Hi Dandreescu.
1) Decide on your TOTAL budget. Don't forget to consider room treatment, speaker stands, cabling, rack, tweaks, dedicated AC lines. Whatever you initially spend is only the ante. You will find the need to make changes and additions as you gain experience. For that reason, it is best if you can gain sufficient knowledge/confidence to buy used. That way, when you are ready to make a change, you can sell something without taking a bath.
2) Mike C. is right on- speakers first. Most importantly, the choice of speakers will determine what kind of amplification you will need. My advice- stick with speakers that have a "benign" impdedance curve (i.e. 8 Amp nominal, without dips below 4 or 5A). This will simplify your life greatly, end up costing you less money, and give you much greater flexibility. I use Spendor speakers (BC1s driven by 30W OTL tube amp)and can't recommend these highly enough. Check out their website (do a google search). This is a company with tradition and standards.
3) You owe it to yourself to listen to tubes. There are very good and inexpensive integrated tube amps that come up on Audiogon with some regularity. One I am thinking of is the Pathos (I've seen it for about $1100). Also consider Antique Sound Labs. Solid state, you could go with a Classe CAP80 for well less than a $1000 (as long as you don't need monster power).
4) DO NOT skimp on the source. If your source fouls things up, there is no way to make up for it. If I was considering how to divy up $4000, I might go with $1800 source (you can get a very good used CD player for this), $1000 integrated, $1000 speakers. Set aside $1000 for the other things you will need.
Good luck, and post progress reports.