Is grounding with RCA is safe?

It’s safe.
Now, there are two complications:

1. as noted you could create a ground loop, but from your report of improved sound, you didn’t, so don;t worry about it.
2. Safety, and effectiveness, depend on how your component is grounded internally. The outside RCA connector is signal ground, for sure. But signal ground and chassis/safety ground may or may not be the same.

• they may be connected internally
• they may be independent internally
• there may be a switch (many of my designs had one to eliminate ground loops)
• They may be connected via a resistor, sometimes around 1-meg-ohm

So, while it won't injure you (its grounded after all), it might or might not provide an improvement.

G

Thank you everyone for your valuable inputs. I never thought that I could get this much help from audio community. 

So if I understand it correctly, what I am doing this is 99% safe. Besides, doing this can compromise the accuracy of the system.

However, I'm concerned about the 1% for my audio gears and safety of people around me. Also, I don't want to mess up with the accuracy of the system. Therefore, I am thinking about a DIY passive grounding box, which is a wooden box with different minerals inside. I think it does not take much time to turn what I am having now into a  DIY passive grounding box. 

Not to take sound quality into consideration, do you think that this solution is safer and less interferes with the accuracy of the system? 

I disagree that grounding options could reduce the accuracy of the system, **unless** they also created a noticeable hum.  no hum, no foul.
The key to grounding is to have ONE connection to ground - and everything else grounded to THAT.  Typically that one point is the preamp.
What I would do is

a) find out if your preamp connects signal and chassis grounds. Ideally float the signal.  Find out the same for your other components.
b) either  b.1) don;t ground any other components or b.2) ground them all to a single power strip/conditioner plugged into a single outlet.
never, ever plug some components into outlet #1 and others into outlet #2. That's the recipe for a ground loop.
Grounding chassis of components to actual earth (3rd plug) is always both safe and good at reducing radiated noise. The problem only occurs if the manufacturer connects chassis to signal grounds.
but I'll re-emphasis --- if it soudns better and has less noise, it almost cannot be creating a loop.
G

The statement that floating chassis from signal ground is always better is a simplistic view that is not always right. Ground hum is not the only form of noise (or distortion) that can be ground induced. That concept totally ignores signal induced noise on the ground that will not be evident when there is no signal and you are just listening to hum.

I am thinking about a DIY passive grounding box, which is a wooden box with different minerals inside. I think it does not take much time to turn what I am having now into a DIY passive grounding box.

Not to take sound quality into consideration, do you think that this solution is safer and less interferes with the accuracy of the system?

This appears to me to be perfectly safe.

As far as the possible sonic consequences are concerned, if the "minerals" are not conductive, and if there are no uninsulated ends of the ground wires making direct contact with each other, it seems to me that all you would be accomplishing is hanging a bunch of antennas off of the circuit grounds of the components.  Which in turn might pick up RFI and conduct it into the components.

If the "minerals" are conductive, or if uninsulated ends of some of the wires are in direct contact with each other, you will have established additional paths between the circuit grounds of some of the components.  Any sonic consequences that may result figure to be unpredictable, IMO.

FWIW, I personally would not consider such an experiment to be a worthwhile investment of time, but that's just me.

Good luck.  Regards,
-- Al