That ORB/Skeleton Architecture on the Server side doesn't look very scalable. What did you leave out?

Almost everything. Figure 1 doesn't show any of CORBA's mechanisms for load balancing, resource control, or fault tolerance on the server side. We deliberately kept the figure simple to demonstrate how CORBA interoperability works.