You're exactly right about that. It's based on a tight interpretation of English. While it's correct, it's not how people talk. People mangle English in different ways to the person. And of course there's national and regional interpretations. So it becomes a matter of what you're writing. I write boring corporate shit for a living and only use "very" for stuff that hits the shop floor. Otherwise I'm translating archaic engineering terms presented by the fossils in product development. For fiction, it's about the character. Would a given character say "we should leave now" or "let's get the fuck out of here". I know a writer who builds a 2 page dossier on each character including odd vocal style and word use. If she hasn't used the character in several chapters, the dossier keeps the delivery in sync. It's like she's spying on characters to give them a broader feel.
I'd love to help, but I'd have to quote from someone else to be able to say something sagacious (thank goodness sagacious was stored in my autocomplete!).