Robot System †

The robot system we use in this hackathon is illustrated in the figure:

It consists of 4-DoF robot arm + 1-DoF gripper (Mikata Arm, by ROBOTIS-Japan), and two FingerVision sensors. The robot is actuated by 5 Dynamixel servos.

Software System †

As the Mikata Arm is actuated by Dynamixel servos, a possible way is using Dynamixel SDK.

As the FingerVision sensors are made with cameras (USB cameras), we can use any computer vision libraries such as OpenCV. The implementations of the marker tracking algorithm and the proximity vision are available from fingervision which provides standalone programs to process the FingerVision data.

However the unification might be complicated with these standalone libraries. Thus, we recommend to use the unified framework ay_tools for the integration over ROS (robot operating system). It is a collection of ROS packages providing Mikata Arm control, simulation, forward and inverse kinematics, and FingerVision integration.

Specifically, we use the following packages.

ay_py

Basic Python libraries including optimization, Graph-DDP, geometry and kinematics calculation, robot control interface of Dynamixel and Mikata arm, ROS utility, etc.
https://github.com/akihikoy/ay_py

ay_cpp

Basic C++ libraries.
https://github.com/akihikoy/ay_cpp

ay_util

ROS utility, including state validity checker (collision checker) over ROS for Python, some launch files, models, etc.
https://github.com/akihikoy/ay_util

ay_trick

Environment for robot system development and experiments. `ay_trick` provides a CUI tool where you can interact with robots. Once the initialization including ROS communication setup is done, you do not have to do it again. You can run external Python scripts.
https://github.com/akihikoy/ay_trick

fingervision

Software for the vision-based tactile sensor FingerVision including CAD models. It has ROS version and standalone version.
https://github.com/akihikoy/fingervision

ay_fv_extra

Extra configurations and launch files of fingervision. This ROS package provides configurations and launch files for specific robots.
https://github.com/akihikoy/ay_fv_extra