The main issue with any electronics is static electricity. Ideally you should use a conductive wristband connected to the chassis. They are not that expensive. I managed for years without one - make sure you discharge any static on yourself by touching the chassis before touching any components. If you are picking up something laying around, it's probably best to ground yourself first and pick up by a part not connected to the circuitry.
If you decide to run the PC with the side off, be careful about temperatures. Although "open" may seem better for dissipating heat, cases are often designed to have certain flow patterns to ensure everything is cooled. Components that were forced cooled by the fans will now be relying on natural convection only.
There are some risks. Part of learning is getting to know how much force is needed. The CPU heat sink for example may need quite a lot of force on the clips. You can only find out how much by trial and error. The error part is not so good! If you are playing with the CPU, it's a good idea to replace the conductive paste while you are at it.
If your memory is not good, write down what is connected where when you take apart!