Good chance that the center bolt/s in the spring pack/s is broken. You would need to take the u-bolts loose enough to see if the center bolt/s has sheared off or not. New bolts are available though your local parts store in the Dorman Products brand line.

The main cause of this happening is that the axle u-bolts were not properly torqued, and re-torqued after the lift kit was installed. If you put a torque wrench on the u-bolts, they should be tightened to about 75 ft lbs if they are 1/2", 55 ft/lbs if they are 7/16". An air impact gun will not fully torque them, use a torque wrench. They will need re-torquing after the first test drive, then after about a week, then after each off-road trip, until you are sure they are holding their torque. New u-bolts stretch out. I recheck them at every oil change on my CJ.

I bet if you go and try to torque the u-bolts (Front and rear) you will find them loose.