Do you have Alum rods ?
When cam grinds came up with larger base circle cams for better lobe shape [ ie, lifter launching on the back side] stanadrd blocks and designs , started hitting the cam with alum rods, especially big rod end rods.
My first JF iron SBC [ bow tie] that I bought from Art Chrisman, he had to have Crane cams grind a SMALLER than OE base circle cam to get enough clearance.
That's why when I built my first SBC, I bought a raised cam block from Dart so I could run the better 55mm cam.
BTW, you don't have to run the roller bearing cam bearings, YA engine builders like them because it turns over easier, but with good oil pressure the cam rolls over as easy as the roller bearings.
Either way you may have to line bore your cam journals larger for you 50mm cam , even with non-roller bearings.
I have heard of some racers having problems with the cam roller bearings moving, but I think thats because they were running super light cranks [ 32 lbs] and the crank was flexing and maybe causing some block flexing. They had to pin the cam bearings.
I never had cam bearing movements in 10 years, of 10k rpm use.
Talking about cams, when a few of our JF racers went with high 8" to 9" valve lift they had some oiling problems, such as no oil going up the push rods, and of course no oil in the rockers, because when using OE style lifter bores, the lifters would lift past the oil passages for the lifters. Matter-a-fact Bob McKray trying to firgure the oil problem, he turned the oil pump over [ on an engine stand] with out the oil pan, and said the oil came out from the cam valley "like a garden hose, so the fix was bore out the lifter bores and install base busings. I never had this problem because I use Jesel Keyway lifters, again for 10 years.
So check your rod to cam clearance before to proceed.