Some believe AI can now output compiled binaries (e.g update Notepad.exe with this feature).

We all think AI writing code for us will be the end, but it might be an even simpler take over.

That just sounds worse though? We can't validate the change is correct if we can't read the code. It is interesting though

Idk what they mean, I've never seen anyone claim, or come close to claiming, it could alter an executable binary by itself. Chose to interpret it as "some people think an llm can code well enough to add features on top of a ~50KLOC codebase automatically"

I think he's saying people believe a LLM trained with this architecture would be able to do something like that.

at some point you can't or won't be allowed to do any validations