Micro-Manipulation Research Projects

• Micro/Meso-manipulation Test-Bed
• Cell Characterization
• Designing Open-loop plans for Planar Micro-manipulation
• Modeling Uncertainty for Planar Meso-scale Manipulation and Assembly
• Planning and Control of Meso-scale Manipulation Tasks with Uncertainties

David J. Cappelleri, Umut Yesilmen, Vijay Kumar, G.K. Suresh Ananthasuresh

• Inverted optical microscope (Nikon Eclipse TEU2000-U, Objectives: 4X, 10X, 20X, 40X)
• 4-axis micromanipulator (Siskiyou Design Instruments MX7600R)
• Controller (Siskiyou Design Instruments MC2000),
• 5 um, 25 um, and 100 um Single Tip tungsten probes (needle), 5 um tip Dual Tip probes
• CCD camera (Sony XC-77)
• 3-axis linear translation stage (Edmund Optics Center Drive 5-inch Square Linear Translational Stage)
• 1 mm OD suction pipette and syringe
• Local Control computer
• Remote Control computer
• Haptic Device (Sensible Technologies Phantom)
• Force Sensors (Transducer Techniques GS-10 Load Cell; MSI, Inc. LT-50 PVDF Film)

Back to top

David J. Cappelleri, Umut Yesilmen, Vijay Kumar

• Objective: 4X
• 1 mm OD holding pipette
• 100 um OD manipulation probe (needle)

Experimental Setup for Cell Characterization Tests

Manipulation tool deforming the cell

Force vs Time plot for successive 20 um steps of manipulation probe causing displacment of cell membrane. After a total displacment of 240 um, probe was stepped in opposite direction, decreasing cell membrane displacement back to zero. The plot shows this test executed two times and the corresponding forces exerted on the cell for each 20 um displacment step as recorded by the load cell.

Due to the visco-elastic properites of the cell, relaxation of the cell membrane occurs when applying a constant force over a period of time. This plot shows this phenomenon for the cell. Over a period of 10 minutes, one can see that the force gradually decreases.

Back to top

David J. Cappelleri, Jon Fink, Barry Mukundakrishnan, Vijay Kumar, J.C. Trinkle

Peg-in-the-Hole Problem
Goal: Re-orientate and move peg from Configuration A to Configuration B

Experimental Hardware
The first generation fixture and peg were manufactured out of mirrored acrylic with a ULS 660 Laser Cutter. The fixture is 1/8" thick and has 2 channels of different widths machined in it. The larger channel is 2015 um wide, while the smaller channel is 1090 um wide. Various size pegs were used; typical dimensions are 1615 um x 985 um. The thickness is half of that of the acrylic fixture.

The second generation fixture and peg were manufacured out of berrylium copper using a photochemical machining process (PCM). The peg and fixture are both 1.5 mils thick. The large channel is 1947 um wide and the small channel is 979 um wide. There is also a playpen area present in the fixture where system identification tests are carried out. Typical dimenstions for the peg are about 888 um x 1600 um.

Modeling Uncertainty for Planar Meso-scale Manipulation and Assembly

David J. Cappelleri, Jon Fink, Vijay Kumar

Abstract:
While robotic assembly at the centimeter and meter length scale is well understood and is routine in the manufacturing industry, robotic grasping and manipulation for meso-scale assembly at the millimeter and sub-millimeter length scales are much more difficult. This work explores a possible way to manipulate and assemble planar parts using a micro manipulator with a single probe capable of pushing parts on a planar surface with visual feedback. Specifically, we describe a study of the uncertainty associated with planar surface friction with a goal of developing a model of manipulation primitives that can be used for assembly. We describe a series of experiments and data analysis algorithms that allow us to identify the main system parameters for quasi-static operation, including the friction coefficient and the force distribution, while characterizing the uncertainty associated with these parameters. This allows us to bound the range of motions resulting from the uncertainty, which is necessary to design robust open-loop meso-scale manipulation and assembly motion plans.

Summary:

Manipulation Model
Due to the scale of the system and speed of manipulation, we assume quasi-static dynamics. We use a simple time-stepping scheme that is specialized to a 2.5D problem where all parts and contact interactions are essentially planar and surface friction in the plane is modeled with a simple force distribution. All contacts are assumed to behave according to Coulomb's friction model.

Specifically, the planar manipulation system is formulated as a mixed linear complementarity problem (MLCP) based on the quasi-static equations of motion and a time-stepping scheme for rigid bodies.

The uncertainty in support pressure distribution and continuous nature of the surface contact makes it difficult to model in simulation. However, it has been shown that with a quasi-static assumption any force distribution can be satisfactorily modeled via a three-point support model model. Alternatively, any motion that results from the application of forces to a part on the plane can be explained by a normal force distribution with three support points using a Coulomb model of friction. Thus if we are given three support point contacts, we can find the normal force distribution based on a out-of-plane force balance, and use the maximum allowable frictional forces resulting from these normal forces in our MLCP formulation. For the purposes of simulation and modeling, the manipulator can be considered an arbitrary convex polygon in the plane with position p_m. Because of our choice to use a three-point support approximating pressure distribution, the part can in fact be any planar polygonal shape.

System Parameters
There are several parameters present in the simulation model that affect the mechanics of the manipulation task and are unknown. These are:

u_s - Coefficient of surface friction
u_t - Coefficient of manipulator-part friction
r_s - A 3x2 matrix specifying the support-point locations

The coefficients of friction u_s and u_t are constrained to the range [0.0, 1.0]. Support-point locations must obviously lie within the dimensions of the part being modeled. Additionally, their convex hull must include the part's center-of-mass so that the appropriate normal force at each support point can be calculated. The set of parameters is an 8-dimensional parameter space. If more contacts are required, the dimensionality of this space increases. Regardless, we can identify the parameter space. The goal is to find the point in the parameter space and the neighborhood of the point that most closely characterize the uncertainty.

Design of Experiments
Eight sets of manipulation tests were used to identify and characterize the parameters in the 8-dimensional parameter space. Manipulation tests consisted of horizontal moves with contact between the probe and the part over distances of approximately 700 um, executed on both the long and short side of the part. The Figures below show the peg and probe in the FOV of the microscope and the schematics of the part in its nominal initial conditions for each set of tests. The coordinate system is chosen to align it with that of the images obtained from the vision system. Pushes were made on the long side of the part at 5 nominal positions - at the midpoint of the side (pt.C), midpoint + l_ls/4 (pt.B), midpoint - l_ls/4 (pt.D), midpoint + l_ls/2 (pt.A), and midpoint - l_ls/2 (pt.E), where l_ls corresponds to the length of the long side of the part. For these pushes, the part was nominally placed at angle of 90°. The short side pushes were located at 3 nominal positions - at the midpoint (pt.G), midpoint + l_ss/2 (pt.F), and midpoint - l_ss/2 (pt.H), where l_ss corresponds to the length of the short side of the part. These pushes start with the part nominally placed at angle of 180°. Each of the starting configurations for the probe in each of the eight manipulation tests are shown in third Figure. A minimum of three trials for each manipulation test were executed.

Peg and probe in microscope FOV

Nominal initial conditions of part for manipulation tests

Staring configurations of probe for manipulation tests

Estimation Algortihm
Given experimental data consisting of trajectories for each of the manipulation test experiments described above, what is the parameter vector p, consisting of u_s, u_t, and r_s, that best explains the experimental data? Our estimation algorithm estimates parameters such that the simulated manipulation test results match the experimental results from the execution of real manipulation tests as well as possible. Because the manipulated part's motion is a non-convex and non-smooth function of the system parameters, we cannot use gradient methods for optimization. This limits us to optimization algorithms that rely only on the evaluation of an objective function at different points.

The objective function we are interested in minimizing is related to the fitting of simulation to experiments over several trial test manipulation runs. For each trial, x_i, and a given parameter vector, p, we compute the simulated motion, s_i, and compute the root-mean-squared error along each axes x, y, theta (with theta scaled by the characteristic length of the part to normalize). This creates a three dimensional error vector that we take the L_infty-norm of to get a quality measure for a single experiment-simulation fit. When fitting across several experimental trials, the total objective becomes simply the average of quality measures across all trials.

Results
At least three trials for each manipulation test were performed. The figure below shows typical trajectories for one trial of each of the eight tests. The starting and ending configuration of the part are shown in solid lines, while the intermediate steps are pictured with dotted lines. The probe tip locations are represented with triangles. The trajectories for each of the trials were not all the same. They were consistent for the most part, but for certain starting configurations small variations from the nominal starting configuration of the part and probe can produce substantial changes in the resulting trajectory.

Snapshots from experimental data for trajectories obtained from selected manipulation tests

Test 1 is an example of a manipulation test that is sensitive to initial conditions, as shown in the figure below. This test involves pushing at the midpoint of the long side of the rectangular part. A horizontal push directly at the center of mass of the part should intuitively result in a pure translation, as seen in the figure for Trial 3. In general, small perturbations from this starting configuration will yield either clockwise or counter-clockwise rotations for the same nominal test as seen in Trials 1, 2, 4, and 5. Most of this can be attributed to errors from the nominal starting position at the beginning of the tests. The accuracy of the vision system is conservatively estimated to be +/- 1 pixel, which corresponds to position measurement errors of the probe and the part of roughly +/- 5 um and angular measurements of roughly +/- $0.005° error.

Trajectories for manipulation test 1

Figure A demonstrates the experimental trajectories of Test 1, that should result in a pure translation but actually produce a wide range of rotations. However, as we can see from Figure B, our model and therefore the simulation predict this sensitivity. When the simulator is presented with initial conditions for the part that lie in a neighborhood of within 5 um and 0.005° of the nominal position, the resulting trajectories exhibit a variation that is similar to that observed experimentally. Because the outcome is more sensitive to small errors in initial onfiguration, it is clear such pathological initial conditions should be avoided for manipulation planning.

The uncertainty of the locally optimal parameter solutions is approximated by the size of the Nelder-Mead simplex which is also the stopping criteria of our implementation and set to 10 um in this case. Specific values for p*_1 and p^_2 are shown in the table and figure below. Although the parameter sets p*_1 and p*_2 are distinct, they are qualitatively similar since they produce trajectories with comparable errors from experimental trajectories.

Estimated Parameter Values

Support point locations for p*_1 and p*_2

The solutions p*_1 and p*_2 are similar but have a large discrepancy in the value of u_s. In fact, we observed that in simulation the value of u_s has minimal affect on the trajectory of the part. This is due to the fact that it is the distribution of support force and not the coefficient of surface friction that will affect the motion. Since we cannot sense the force being applied by the probe, we cannot observe the actual frictional forces being exerted.

Our results are the first step toward creating models for quasi-static, vision-guided meso-scale manipulation. Indeed preliminary results show that this approach is promising for simple manipulation tasks. An example of the application of this approach is shown below. The plan for assembling the part was generated automatically based on the model derived in this work and the snapshots shown in the figure were taken during the execution of this plan. Because the clearance between the 1612 um x 842 um rectangular part and the slot is 137um, it is possible to use our approximate model for successful assembly. Our approach yields a 100% success rate in such tasks. However, it is clear that tighter tolerance tasks will require better models.

Reference

• D.Cappelleri, J. Fink, V. Kumar, Modeling Uncertainty for Planar Meso-scale Manipulation and Assembly, Submitted to ASME 2006 International Design Engineering Technical Conference (IDETC), Philadelphia, 2006.

Back to top

Planning and Control of Meso-scale Manipulation Tasks with Uncertainties

Peng Cheng, Bogdan Gavrea, David J. Cappelleri, Vijay Kumar

Abstract:
We consider the canonical problem of assembling a peg into a hole using probes at the end of a micro manipulator using open-loop pushing operations with feedback between the pushing operations from an optical microscope. We develop a quasi-static model for the assembly task incorporating models of frictional contacts and consider three sources of uncertainty. Because of errors in sensing position and orientation of the parts to be assembled, we must consider uncertainty in initial configuration of the system. Second,there is uncertainty because of errors in actuation. Third, there are geometric and physical parameters characterizing the environment that is unknown. We discuss the synthesis of robust planning primitives for meso-scale manipulation using two different probes and the automated generation of plans for manipulation. We show simulation and experimental results in support of our work.

Summary:

Introduction
The manipulation problem considered here is governed by differential equations subject to unilateral constraints and
uncertainties. We are interested in synthesizing open-loop controls or motion plans for such systems.

The Control System
Assume that the motion of the control system in the given environment is characterized by

x_dot = f(x,u,p)

in which x ∈ X ⊂ Rⁿ is the state, u ∈ U ⊂ R^m is the input, and p ∈ P ⊂ R^l is the parameters for the system and environment. We use U to denote the control space, which includes all controls for the system. A motion m is fully characterized by (x₀, u, p). We will denote the set of all motions by M.

Uncertainties
We consider three bounded uncertainties stemming from sensing, control (actuation), and the environment.

Sensing uncertainty
We assume sensors that can estimate the global state of the system with bounded error s^u_x with a sensing rate s^r (the sensing is
carried out every 1/s^r second). We will use x and x^s to respectively represent the actual and sensed states of the system. Therefore, we have

x ∈ B_s^u_x (x^s)

in which B_r(x') = { x | ||x, x'|| ≤ r} is the r-neighborhood of state x with respect to a metric ||·,·|| on X.

Control uncertainty
We assume that actuators will realize the commanded control with a bounded error c^u_u with a control rate c^r (the minimum time
between two control commands is 1/c^r second).

We will use u and uⁱ to respectively represent the actual and intended controls for the system. Therefore, we have

u ∈ B_c^u_u(uⁱ)

Environment uncertainty
We assume that the environment p-history is parameterized by p with bounded error e^u_p. We will use p and pⁿ to respectively represent the actual and nominal p-history for the environment. Therefore, we have

p ∈ B_e^u_p(pⁿ)

Goal
Given a sensed initial state x_init and a goal set X_goal = B_tau(x_goal), the objective is to compute a control u (that may depend on feedback information) which will drive the system from x_init to X_goal under uncertainties.

To solve the above problem is quite difficult. Because complete algorithms are difficult to find except for the simplest of problems, we pursue the synthesis of plans that are obtained by composing robust motion primitives.

Robust Motion Primitives
A robust motion primitive for a control system in a given environment is defined as a control such that the resulting motion m preserves a given property v in the presence of uncertainty. Assuming that a motion m' = (x'₀,u', p') is in the uncertainty neighborhood of motion m = (x₀, u, p), we can take the property v as a function, v: M --> {0,1}. If v(m)=1, then we say that the motion m satisfies the given property and is called a v-motion.

The admissible set, denoted as A_v,for a v-motion is defined as: A_v = {m'| v(m')=1}

If m is a v-motion, we can ask if another motion in the neighborhood of m also satisfies the property v. We denote by E^u=(E^u_x, E^u_u, E^u_p), the size of the maximal neighborhood of m in the admissible set containing these property-preserving motions (see figure below). Because any motion is characterized by the uncertainty e^u=(s^u_x, c^u_u, e ^u_p), a v-motion is a robust motion primitive only if its e^u-neighborhood is contained within its E^u-neighborhood.

Robust motion primitives

If there exists a robust v₁-motion and a robust v₂-motion such that the v₁-motion can be reliably appended to the v₂-motion under uncertainties, then we say that it is possible to sequentially compose the motion primitives.

Thus our approach to planning will involve the construction of a set of robust motion primitives followed by their sequential composition. At this point, a graph search algorithm can be used to synthesize the complete motion plan. It is worth mentioning that such algorithms are not complete because they restrict the search space from the original control space to a smaller one consisting only of robust motion primitives.

Properties of Motions
For our meso-scale manipulation task involving pushing primitives, the different contact states (including sticking, sliding to the left, sliding to the right, contacts on the same edge or different edges, and their combinations with respect to either the Single Tip Probe (STP) or the Dual Tip Probe (DTP) between the peg and probe are all different properties which we can seek to maintain or keep invariant. Some special properties for these types of motions follow. The first property is for the STP or for a DTP with single contact to rotate the object in a specified direction (clockwise or counter clockwise) while maintaining sticking contact. The second property is for the DTP to maintaining two-point sticking contact. These two properties will be combined to define a to-two-contact property, denoted as t², which is a sequential composition of a one-point sticking contact motion with intended rotation followed by a two-point contact with a two-point sticking contact motion.

Examples of robust motions

System Dynamics
We will use a quasi-static model for the system (inertial forces are of the order of nano-newtons for the accelerations involved, while the frictional forces are on the order of micro-newtons). The friction between the probe and peg surface is characterized by Coulomb's law with friction coefficient mu. We assume the support
plane to be uniform, and all pushing motions of the probes to be parallel to this plane. The most important assumption is about the support friction. Because we coat the support surface with oil, it is reasonable to assume viscous damping at the interface. Based on experimental data we chose the model

f= E v

where v = [v_x, v_y, v_theta]^T is the velocity ([v_x, v_y, and v_theta are the velocity components of the centroid of the
peg), f= [f_x, f_y, m_theta]^Tis the corresponding vector of forces and moments, and E is the diagonal, 3x3 damping matrix. It has the elments e_x, e_x, and e_theta along the main diagonal and zeroes everywhere else. These three parameters along with coefficient of friction are fit with experimental data.

Finally, we assume the only contacts that occur are with the probe and the object. Although we consider the assembly task as our goal, we only consider the problem of guiding the peg into the designated slot without any collisions with the environment.

Robust Motion Comparisons
Tthere might exist many types of robust motions with respect to different invariant properties. We provide a measure, Lipschitz constant of the motion equation, to compare robustness of these motions with respect
to uncertainties.

The Lipschitz constants have been used before to provide an upper bound on the variation of the trajectory with respect to changes in the state, control, and parameters. The magnitude of Lipschitz constants characterize the worst case behavior of the system under uncertainties. If the Lipschitz constant is smaller, then the upper bound on the trajectory variation with respect to uncertainties is smaller, i.e., the corresponding motion will be more robust.

The following theorem states that the DTP will have less uncertainty than the STP with this measure, which is supported by the experimental results shown later.

• Theorem: Assume that both the STP and DTP are used to push a peg at a fixed configuration with same probe initial position (the top tip for the DTP and the tip of the STP have the same position), constant fixed velocity, and time duration. Both probe pushing starts with no contact and has the same invariant contact mode. Under the same variation in the initial peg configuration x₀, y₀, theta₀, pushing position d₂, and parameters E (damping matrix) and possibly mu (friction coefficient), the DTP pushing will induce no more trajectory variation than the STP pushing.
  

Algorithm Description
Our algorithm relies on the design of three robust motion primitives which are enough to complete the given assembly task. These are as follows:

1. Robust translation in the x-direction
   With the DTP, translational in the x-direction will be achieved by applying the robust t² motion primitive in the x-direction.
2. Robust translation in the y-direction
   Robust translation in the y-direction is achieved by composing a robust rotation motion and a robust t² motion. The amount of the net vertical translation is l_AB(1-cos(phi)) under nominal conditions (no uncertainty).
   
   
   
   Vertical translational motion
   
3. Rotational motion
   This motion will be achieved by pushing with two STP with a separation: d_v > d_w + 2 s^u_x + 2 c^u_u
   to guarantee the intended rotation under sensing and control uncertainties.
   
   
   
   Rotational motion conditions
   

The planning problem can now be setup according to the figure below. The the initial configuration with orientation π / 2 and goal configuration of (0, 0, 0) with a position and orientation tolerance of ε _p = 0.06 and
ε _theta = π / 2, respectively.

Setup for the planning problem

The planning algorithm then consists of the following steps:

Step 1. Move in the y-direction such that y ∈ [-ε _p/ 2 , ε _p / 2]

Step 2. Rotate to the theta = π configuration

Step 3. Translate the peg in the x-direction to the goal (0,0,0)

Design of Experiments
A set of 5 manipulation tests were designed and executed on the peg with both the DTP and STP in our micromanipulation test-bed. The DTP in the microscope field of view (FOV) as well as schematics for the manipulation tests can be seen below.

Test pushes were executed in the X-direction, between 700 um and 1000 um in length. For comparison, the vertical position of the STP corresponds to the vertical position of the mid-point, top or bottom of the
DTP in the prescribed tests.

Dual Tip Probe (DTP) and peg in microscope FOV

At least three trials were executed for each manipulation test where the peg and the probe trajectories were calculated and recorded. The figures below show the snapshots of the trajectories of the peg and and probes for one trial of each of the manipulations tests. The triangles represent the position of the tip of the probe. For the DTP, there are two such tips shown.

Estimation of System Parameters
The parameter fitting is obtained by using experimental data from the manipulation tests. The figure below shows experimental trajectories versus predicted trajectories for one trial that was used in the parameter estimation (left) and one trial that was not (right). To estimate the parameters a root-mean-square metric is used. The optimization algorithm we are using is based on the Nelder-Mead searching method.

The estimated system parameters are listed in the following table:

Comparison Between the Robust and Un-robust Motions
The figure below shows the experimental trajectory plots for comparison of the robust and un-robust motions using the DTP and STP. Tests 1 and 2 are for robust and un-robust t² motions with the DTP. Test 1was verified to satisfy the robust t² motion conditions. The experiments showed that the two-point contact is well maintained because the orientation, theta, is almost constant after the two point contact is established. Test 2 did not satisfy the two-point sticking contact conditions, and therefore the two point contact was broken once it was established. We also observed that Test 1 has maximal trajectory differences of 20 um in x, 15 um in y, and 0.023 radians in theta, which is smaller than Test 2 (maximal trajectory differences at 15 um in x, 25 um in y,and 0.1 radians in theta).

Comparison Between DTP and STP
Tests 1 and 3, shown above are robust motions, respectively for the DTP and STP. The top tip of the STP had the same y-position as the DTP. We observed that the motions from Test 1 have less uncertainties than those from Test 3, whose maximal trajectory differences are 75 um in x, 75 um in y, and 0.2 radians in theta. Therefore, for comparable robust motions, the DTP yields more accurate results than the STP.

Movies
Click anywhere in window to activate movie and then on triangle at bottom left of image to play it.
(Works best with Internet Explorer)

Note: In these tests, the part is being manipulated on thin layer of oil. The location where the probe tips penetrate the film is observable in the captured images.

Test 1: DTP Robust t² motion
Movie of trial from DTP-Test 1

Test 2: DTP Un-robust t² motion
Movie of trial from DTP-Test 2

Test 3: STP Robust motion
Movie of trial from STP-Test 2

Planning in Both the Simulation and Experiment
The uncertainty bounds in the sensing and control in computing the robust motions are s^u_p = 10 um, s^u_theta = 5°, and c^u_p = 5 um according to the precision of the image processing. These uncertainty bounds also satisfy the conditions to ensure the required task precision. The figure below shows the simulation results for a planned motion for the peg at the initial configuration (2.5, 0.05, π / 2).

The snapshots of the experimental verification of plan are shown here with corresponding controls from the simulation.

Step 1: Move peg in the y-direction



Step 2. Rotate to the Theta = π configuration



Step 3. Translate the peg in the x-direction to the goal

Reference

• P. Cheng, D. Cappelleri, B. Gavrea and V. Kumar. Planning and Control of Meso-scale Manipulation Tasks with Uncertainties, Proceedings of Robotics: Science and Systems (RSS), Atlanta, GA, June 27-30, 2007.

Back to top