That's not how it works now, except for music. From the wikipedia article on mechanical licenses (the link I provided):

"Within copyright law within the United states, such mechanical licenses are compulsory; any party may obtain a license without permission of the license holder by paying a set license fee, that as of 2018, was set at 9.1 cents per composition or 1.75 cents per minute of composition, whichever is more, which are to go to the composition copyright holder."

Note "compulsory". There are a multitude of written and art works that are effectively copyright zombies: they are clearly within the term of copyright, but there is no clear owner to reach out to in order to license them. Hence generally out of fear of a lawsuit, they are dead as inspirations for other works.

Further, to your exact point, "if they so desire. The fee is determined by your legal team and theirs." If the author wants to be restrictive, their work will never be able to be inspirational to another work. Even if they are open to the idea, the concept that lawyers (probably) have to get involved reasonably has to diminish the number of works that are likely ever to be inspirational for other works by an enormous number.

The difference is the (generally) required licensing and the set fee.

For example, as I understand it, a playwright can basically say "I don't let high schools or colleges stage my works." With respect to book adaptations, there were actually some issues with Sorkin's script for To Kill a Mockingbird because he made some changes in Atticus Finch's character development relative to the book.