Lazy Heartbeat Scheduling

[NthTensor]

At the start of the year, I started circulating this crate about and soliciting feedback. The general response was: it doesn't perform well. Womp womp. Sad trumpet noises.

Over the last few months, I tried built several prototypes aimed at improving performance. This PR is the culmination of those efforts. Some of of the intermediate experiments got pretty pretty messy, so for simplicity I've chosen to just pull the working experiments together into this Mega-PR.

There are three major changes:

• I've added a new lock-free worker registration and sleep-tracking system (Basically we use an atomic bit-set for tracking seat information). This has significantly improved the overhead of worker wakeups, and allows us to wake workers up much more eagerly, but also means thread pools now have a hard limit of 32 members. I think this is a reasonable constraint (and we can theoretically expand it up to 64 on platforms that support 64-bit atomics).

• There's a totally new (potentially novel) approach to workload balancing that combines work-stealing, heartbeat scheduling, and lazy scheduling. This is detailed in the new readme, so I won't repeat it here.

• I've also done another comprehensive pass on docs and safety comments.

This code will form the basis for 1.0.0-alpha.5 and (depending on how the next round of feedback goes) may end up being released as 1.0. Hopefully changes will be more incremental going forwards.

[CorvusPrudens]

Here are my results. My system isn't particularly quiet, so take it with a grain of salt.

https://gist.github.com/CorvusPrudens/16c4f6dc26770c8578a869978eb85ab8

[NthTensor]

There seems to be a potential segfault within join on x86 windows, which I am trying to diagnose.