Yes, you got that right. Keeping it in an exchange is more risky than on a computer.
But a wallet on a computer is still more risky than an offline wallet, because your computer can be hacked, stolen or destroyed. That's why an offline hardware wallet is the best solution.

It's just that @arqetype mentioned he stored his coins at an exchange for now because he doesn't hold much cryptocurrency. That's why I said it's fine to do that for now, but in the future a hardware wallet will be better.