Problems with starter engine

"I have checked the starter relay, sometimes it gives 12v and sometimes it gives 4 or 6v"
What are you checking? The red cable from the battery should always be 12 V while the black cable to the starter should be 0 V except when cranking, when it should be around 11 V. You have a bad electrical contact if otherwise. The Y/G to the solenoid coil should be 0 V when not cranking and 12 V when the start button is pressed. The B/G lead from the solenoid to the clutch switch should be 0 V when not cranking and 0 V or 12 V when the start button is pressed, depending on whether the clutch lever is pulled or not.

Measure the voltage at the starter's cable connection when you press Start.

The ground cable connects to the crankcase near the starter and to the battery on the other end. Check both ends for tightness. Will the bike push start?

You could loosen the bolt attaching the cable and wiggle the terminal back and forth before retightening. But removing and cleaning sounds like overkill unless it's covered with corrosion and/or grime. I'd normally expect the solenoid to be bad but you have 12 V at the starter and have confirmed that the starter works. At that point I begin to wonder about something being seized or jammed in the starter gear train or in the engine itself. The starter brushes might be bad but you said that it will turn when removed.