The paper presents a robotic system for object manipulation based on information obtained from a camera. The developed system enables the differentiation of four classes of objects with regular geometric shapes. To achieve that, a semantic segmentation model was trained using a set of images of objects in different positions. An algorithm for objects’ position and orientation determination was developed so objects can be placed in arbitrary positions and orientations within the camera's field of view. The developed algorithms ensure the necessary information for automatic robot programming for moving the objects to desired poses. To prove the proposed concept on the 4-axis SCARA robot equipped with a vacuum gripper for object grasping, a camera calibration procedure was performed and necessary coordinate transformations were obtained. The verification of the developed system was conducted through several experiments. The experiments showed good reliability of the trained model for objects’ classification and accurate positioning of the robot end-effector above the objects.