Advice needed: am I going in the right direction?

I'm new to hi-fi sound world and spontaneously bought myself Yamaha NS BP200 passive speakers and Onkyo A-9030 amplifier, because I thought that connecting it to a PC will be a piece of cake - just buy some external sound card and you are good to go!

However after doing some research I found out that it's not that simple and now I'm kinda lost: does it (PC -> DAC or sound card -> amp -> speakers) even work like this. So, my question is:

Am I going in the right direction? And if yes, maybe you can suggest some DAC/external USB sound card that could suit my case (budget is about $150)?

I would get an Audioquest Dragonfly.  It is a DAC that looks like a usb stick and fits your budget.  You will need a cable that has a mini jack to fit into the DAC and terminates in RCA inputs for your amp
You are correct. Connect computer to DAC, DAC to amp, amp to speakers. Computer needs to recognize DAC, may need a driver depending on DAC and type computer. 
If using USB out from computer and you want to keep it simple and inexpensive there are HRT Music Streamer DACs for sale on Ebay, actually one listed for $57($5 shipping) that would get you started. I have used these in several budget systems and they worked well. Another option is the Schiit Audio Modi 3 which would provide more input options for around $100 shipped.

Good luck 
Thank you so much, guys. 

One thing that is now not that clear for me, is there any difference between a cheap DAC and an expensive one, except for some noise that could be present in a low-priced ones and amount of outputs? Will they produce the same signal for the amplifier?

I'm an amateur musician myself and also wanted to buy the Steinberg UR12 external sound card to make some recordings on it. But will it also work for converting digital to analog for my sound system as an external DAC and will there be a difference in sound between Steinberg and some $100 DAC?

Just like all the other components in an audio system, DACs will have different sonic signatures.  Some are "warm", some are "neutral", some are "bright", some are more detailed, etc. etc. etc.

At the level your system is currently at those differences are probably going to be less obvious.  The more revealing your system is, the more those differences will be noticeable and the more they will matter.