That was such a frustrating decision. I had almost convinced Spotify that that one time I listened to Lustmord was just a random mood, and I don't actually want to only listen to dronecore for the rest of my life.
I don't know those terms and now I'm afraid to search for them. The Cybernetic Bureaucracy Mind might label me a dissident with terrible taste in music.
I think it's something people keep rediscovering. It's a pretty fun programming problem that lets you explore lots of different domains at the same time (video processing, color theory, different coordinate systems for visualizing things) and you get a tangible "cool" piece of art at the end of your effort.
I built one of these back in the day. Part of the fun was seeing how fast I could make the pipeline. Once I realized that FFMPEG could read arbitrary byte ranges directly from S3, I went full ham into throwing machines at the problem. I could crunch through a 4 hour movie in a few seconds by distributing the scene extraction over an army lambdas (while staying in the free tier!). Ditto for color extraction and presentation. Lots of fun was had.
I'd argue it has less to do with the "capitalist systems" and more to do with how obnoxious "slugging" sounds. Commuting in your own car guarantees (a) you have a ride and (b) you have flexibility. You can change you plans, stop by the store, run errands, anything your heart desires (plus, not deal with strangers everyday. Ugh.)
This isn't a pro-car thing. I've haven't driven a car to work in 8 years (I pay out the ass to live downtown so that I'm close enough to walk / bike). It just seems like "Well, that sounds like a pain in the ass" is a simpler possible explanation to why slugging isn't popular in the US compared to Big Business not wanting it to be.
Hard disagree. I think a huge part of our job as engineers is to build systems that can outlive us and comfortably change hands (without the next team cursing the former).
Maybe this is born from spending so many years in Amazon (with it's high turn over and near-quarterly re-org shuffling), but what's getting called "replaceable" here I'd call "writing maintainable software."
The goal is to get knowledge out of your head and into the codebase so everyone can reap the benefits. Knowledge hoarding is lame.
Nothing, I suppose. I honestly didn't realize it's so contentious. I guess it just seemed kind of weird for "the news" to have an opinion at all. Why do people want an organization to tell them who they think should be president?
I know nothing about snake hunting other than the 3 youtube videos I just watched (https://www.youtube.com/shorts/5JB_bzQwtZ8), but this professional sure makes getting them into the sack is about as hard and dangerous as I would imagine. Seems like you're one slip up from getting a chunk of your body removed (like that dude in the youtube video's arm).
Most of the pythons they are bagging are much smaller than that, and YouTubers have an incentive to make bagging a snake look more thrilling than it usually is.
This question is probably obvious if I knew what a microbatch or topology or depot was, but as a Rama outsider, is there a good high level mental model for what makes the cross-partition transactions work? From the comments that mention queuing and transaction order, is serialized isolation a good way to imagine what's going on behind the scenes or is that way off base?
A depot is a distributed log of events that you append to as a user. In this case, there's one depot for appending "deposits" (an increase to one user's account) and another depot for appending "transfers" (an attempt to move funds from one account to another).
A microbatch topology is a coordinated computation across the entire cluster. It reads a fixed amount of data from each partition of each depot and processes it all in batch. Changes don't become visible until all computation is finished across all partitions.
Additionally, a microbatch topology always starts computation with the PStates (the indexed views that are like databases) at the state of the last microbatch. This means a microbatch topology has exactly-once semantics – it may need to reprocess if there's a failure (like a node dying), but since it always starts from the same state the results are as if there were no failures at all.
Finally, all events on a partition execute in sequence. So when the code checks if the user has the required amount of funds for the transfer, there's no possibility of a concurrent deduction that would create a race condition that would invalidate the check.
So in this code, it first checks if the user has the required amount of funds. If so, it deducts that amount. This is safe because it's synchronous with the check. The code then changes to the partition storing the funds for the target user and adds that amount to their account. If they're receiving multiple transfers, those will be added one at a time because only one event runs at a time on a partition.
To summarize:
- Colocated computation and storage eliminates race conditions
- Microbatch topologies have exactly-once semantics due to starting computation at the exact same state every time regardless of failures or how much it progressed on the last attempt
My entire team is in another time zone. And like many, I don't even have an assigned desk at the office. So, my RTO is commuting in, booking an empty conference room, and then dialing in to meetings with people on the other side of the country.
reply