I work in an office, very sedentary work.
I had surgery on a Tuesday and returned to work by the following Monday (6 days).
I'd say that if you have a relatively sedentary job, you could probably return relatively quickly. However, just because you CAN doesn't mean you should. I know that I had pretty bad fatigue for at least 3 weeks after surgery. In my 1st week back, I fell asleep at lunch one day. And for several weeks, I crashed every Thursday and fell asleep in the chair right after dinner. If I had to do it again, I probably would have taken a 2nd week off.