USOO76893.

20B2

(12) United States Patent (10) Patent No.: US 7.689,320 B2

(54) ROBOTICSURGICAL SYSTEM WITH JOINT 2002/0159606 A1* 10/2002 Hobelsberger ............... 381.96
MOTON CONTROLLERADAPTED TO 2003/0004610 A1* 1/2003 Niemeyer et al. . ... 700,245
REDUCE INSTRUMENT TIP VIBRATIONS 2003/0018400 A1 1/2003 Tuttle et al. ................... TOO,29
2004/0128030 A1* 7/2004 Nagata et al. ..... ... 700,245
75 2004/0174770 A1* 9, 2004 Rees ............................. 367/7
(75) Inventors: Step
s
E,Mt.
s
Es)
s
2004/0238758 A1* 12/2004 Antonius Theodorus
Dams ...................... 250/491.1
O O 2005, 0021176 A1 1/2005 Takenaka et al. ............ TOO.245
(73) Assignee: Intuitive Surgical Operations, Inc., 2005/0259348 A1* 1 1/2005 Zhang et al. ............. 360/77.02
Sunnyvale, CA (US) 2006/0293791 A1* 12/2006 Dariush ...................... TOO.245
- 2007/00 1333.6 A1 1/2007 Nowlin et al. .......... 318.568.21
(*) Notice: Subject to any disclaimer, the term of this
patent is extended or adjusted under 35 OTHER PUBLICATIONS
U.S.C. 154(b) by 762 days.
Vertut, Jean and Coeffet, Philippe Coiffet: “Robot Technology; vol.
(21) Appl. No.: 11/509,172 3A Teleoperation and Robotics Evolution and Development'; 1986;
9 Prentice-Hall, Inc; Englewood Cliffs, N.J.
(22) Filed: Aug. 24, 2006 (Continued)
(65) Prior Publication Data Primary Examiner Khoi Tran
US 2007/O142968 A1 Jun. 21, 2007 Assistant Examiner—Rodney King
Related U.S. Application Data (57) ABSTRACT
(60) Provisional application No. 60/751,947, filed on Dec. A robotic Surgical system has a robot arm holding an instru
20, 2005. ment for performing a Surgical procedure, and a control sys
tem for controlling movement of the arm and its instrument
(51) Int. Cl. according to user manipulation of a master manipulator. The
G06F 9/00 (2006.01) control system includes a filter in its forward path to attenuate
(52) U.S. Cl. .................. 700/245; 700/264; 318/.568.1 1. master input commands that may cause instrument tip vibra
318/.568.12 tions, and an inverse filter in a feedback path to the master
(58) Field of Classification Search ................. 700/245, manipulator configured so as to compensate for delay intro
7OOf 264 duced by the forward path filter. To enhance control, master
See application file for complete search history. command and slave joint observers are also inserted in the
(56) References Cited control system to estimate slave Joint position, velocity and
acceleration commands using received slave joint position
U.S. PATENT DOCUMENTS commands and torque feedbacks, and estimate actual slave
ck joint positions, Velocities and accelerations using sensed
4,068,156 A 1/1978 Johnson et al. ............. 318/.575 slave joint positions and commanded slave joint motor
4,883,400 A * 1 1/1989 Kuban et al. ................... 414/2 tOrcules
5,252,901 A * 10/1993 Ozawa et al. .......... 318.568.12 CueS.
6,364,888 B1 4/2002 Niemeyer et al.
2002/01 12855 A1 8, 2002 Arndt et al. ............ 166,250.15 25 Claims, 9 Drawing Sheets

-100
 US 7.689,320 B2
Page 2

OTHER PUBLICATIONS Blake Hannaford, Proc. of the 2002 IEEE Int. Conf. on Robotics &
Automation, Washington DC, USA, pp. 1863-1869.
Y. Yokokohji T. Yoshikawa Bilateral Control of Master-Slave J. Rosen, B. Hannaford, M. MacFarlane, M. N. Sinanan ; Force
Manipulators for Ideal Kinesthetic Coupling- Formulation and Controlled and Teleoperated Endoscopic Grasper for Minimally
Experiments; IEEE Transaction on Robotics and Automation vol. 10 Invasive Surgery- Experimental Performance Evaluation; IEEE
No. 5 Oct. 1994. Transactions on Bimedical Engineering vol. 46, No. 10 Oct. 1999.
A.Sherman M.C.Cavusoglu, F.Tendick.Comparison of Teleoperator P.J. Hacksel, S. E. Salcudean; "Estimation of Enviroment Forces and
Control Architectures for Palpation Task; IMCE’00 Symp on Haptic Rigid-Body Velocities using Observers'; IEEE International Confer
Interfaces for Virtual Environment and Teleoperator Systems; Nov. ence on Roboticxs and Automation San Dier, CA 1994.
5-10; Orlando FL. R. V. Dubey, T.F. Chan, S.E. Everett; Variable Damping Impedance
J. Funda, R. Taylor, B. Eldrige, S.Gomory, K. Gruben Constrained control of a Bilateral Telerobotic System Control Systems Magazine,
Cartesian Motion Control for Teleoperated Surgical Robots IEEE IEEE: Publication Date: Feb. 1997 vol. 17, Issue: 1; pp. 37-45.
Transacations on Robotics and Automation vol. 12 No. 3 June 1996.
R. Anderson M. Spong:Bilateral Control of Teleoperators with Time K. H. Zaad, S.E. Salcudean :"Analisys and Evaluation of Stability
Delay; IEEE Transacton on Automatic Control vol. 34 No. 5 May and Performance Robustness for Teleoperation Control Architec
1989.
tures'; Proceeding of 2000IEEE International Conference on Robot
M.C. Cavusoglu, F. Tendik, M. Cohon, S. Sastry: A Laparoscopic ics and Automarion, San Francisco CA. Apr. 2000.
Telesurgical Workstation; IEEEE Tranactions on Robotics and Auto Y. Yokokohji, T. Imaida, T. Yoshikawa ; “Bilateral Control with
Energy Balance Monitoring Under Time-Varying Communication
mation vol. 15 No. 4 Aug. 1999. Delay” proceeding of 2000IEEE International Conference on Robot
F.Tendick, M. C. Cavusoglu, N. Dhruv, A. Sherman; “Maximazing ics&Automation San Francisock CAApr. 2000.
the Sensation of Compliance in Teleoperative Surgery.” 8' Interna F. Feffard, E. Andriot; A. Micaelli; On the Use of a Base Force?Torque
tional Symposium on Robotics With Aplications (wac 2000) Jun. 16, Sensor InTeleoperation Proceeding of 2000 IEEE International Con
2000.
Bilateral controller Design for Telemanipulation in Soft Environ ference on Robotics & Automation; San Francisco, CA Apr. 2000.
ments; M. C. Cavusoglu, A. Sherman, F.Tendick; IEEE International Speich; Joehn E.; Fite, Kevin; Goldfarb, Michael; “A Methof for
Conference on Robotics and Skutomation(ICRA2001), Seoul, Korea Simultaneously Increasing Transparencey and Stability Robustness
May 21-26, 2001. in Bilateral Telemanipulations'; Proceeding of 2000 IEEE Interna
Keyvan Hashtrudi-Zaad & Salcudean; Septiuiu; Bilateral Parallel tional Conference on Robotics & Automation; San Francisco, CA
Force/Position Teleopertion Control; Journal of Robotic System vol. Apr. 2000.
19 Issue 4 pp. 155-167. Teleoperation Based on the Adjoint Jacobian Approach Y. Tsumaki,
S.E. Salcudean, S. Ku, G. Bell; Performance Measurement in Scaled D.N. Nenchev, S. Kotera, M. Uchiyama;: Control Systems Magazine,
Teleoperation for MicroSurgery Proceedings of the First Joint Con IEEE: Date: Feb. 1997 vol. 17, Issue: 1; 53-62.
ference on Computer Vision, Virtual Reality and Robotics in Medi B. Hannaford A:"Design Framework for Teleoperators with
cine and Medial Robotics and Computer-Assisted Surgery; 789-798; Kinesthetic Feedback. IEE Transactions on Robotics and Automa
1997. tion, vol. 5 No. 4;Sep. 1989.
K. H. Zaad, S. E. Salcudean:Transparency in Time-Delayed Systems Jaydeep Roy, Lous L. Whitcomb:"Adaptive Force Control of Posi
and the Effect of Local Force Feedback for Transparent Teleopera tion/Velocity Controlled Robots: Theory and Experiments' Robotics
tion; IEEE Transaction on Robotics and Automation vol. 18 No. 1, and Automation, IEEE Transactions; Apr. 2002vol. 18, Issue: 2 pp.
Feb. 2002. 121-137.
W.H. Zhu, S.E. Salcudean :"Teleoperation with Adaptive Motion? Jaydeep Roy, D. Rothbaum, Louis Whitcomb Haptic Feedback Aug
Force Control' 1999 IEEE International Conference on Robotics and mentation through Position Based Adaptive Force Scaling: Theory
Automation, May 10-15, 1999, Marriott Hotel, Renaissance Center, and Experiments 2002 IEEE/RSJ Intl Conference on Intelligent
Detroit, Michigan, Proceedings. IEEE Robotics and Automation Robots and Systems EPFL Lausane, Switzerland; Oct. 2002.
Society 1999, vol. 1, p. 231. G.M.Prisco, M.Bergamasco Preliminary Considerations on the
D.W. Repperger; Active Force Reflection Devices in Teleoperation; Design of Controllers for Haptic Interfaces Proceedings of
IEEE vol. 11, Issue 1, Jan. 1991 pp. 52-56. SPIE vol. 3206 Telemanipulator and Telepresence Technologies
Murat Cenk Cavusoglu ; Control of a TeleSurgical Workstation Uni IV. Matthew R. Stein, Editor, Dec. 1997, pp. 2-11.
versity of California, Berkeley Technical Report No. UCB/ERL J. K. Salisbury; "Active stiffness control of manipulator in cartesian
M97/35 1997. coordinates' 19'IEEE Confererenco on Decision & Control; 1980
D. A. Lawrence:Stability and Transparency in Bilateral Teleopera 95-100.
tion IEEE Transaction on Robotics and Automation, vol. 9 No. 5 Oct. C. Natale R. Koeppe: "An Automatic Procedure for Force Controller
1993. Design”: dilr Oberpfaffenhofen, Institute Report IB 515-99-05, 1999.
T. Itoh, K. Kosuge, T. Fukuda Human-Machine Cooperative R. Volpe, P. Khosla: “The Equivalence of Second-Order Impedance
Telemanipulation with Motion and Force Scaling Using Task-Ori Control and Proportional Gain Explicit Force Control” The interna
ented Virtual Tool Dynamics IEEE Transactions on Robotics and tional joural of robotics research, vol. 14 No. 6 Dec. 1995, pp.
Automation vol. 16 No. 5 Oct. 2000. 574-589.
P. Arcara, C. Melchiorri, S. Stramigioli:"Intrinsically Passive Con Murat Cenk Cavusoglu, Alana Sherman, Frank Tendick; “Design of
trol in Bilateral Teleoperation Mimi Systems' Proceedings of the Bilateral Teleoperation Controllers for Haptic Exploration and
European Control Conference 2001 p. 1180-1185. Telemanipulation of Soft Environments' Robotics and Automation,
S.E. Salcudean, M. Zhu, W. Zhu, K. H. Zaad; Transparent Bilateral IEEE Transactions; Aug. 2002:vol. 18, Issue: 4641-647.
Teleoperation under Position and Rate Control. The International S.E. Salcudean, W. H. Zhu, P. Abolmaesumi, S. Bachmann and P.D.
Journal of Robotics Research, vol. 19, No. 12, 1185-1202 (2000) Lawrence; "A Robot System for Medical Ultrasound” Department of
DOI: 10.1177 O278.3640O22068020. Electrical and Computer Engineering . . . www.ece.ubc.ca/-tims/
W. Zhu, S. E. Salcudean ; Stability Guaranteed Teleoperation: an pubs/ISRR99Salcudean.pdf.
Adaptive Motion/Force Control Approach IEEE Transactions on vol. Alin Albu-Schaffer, Gerd Hirzinger: “Cartesian Impedance Control
45, Issue 11, Nov. 2000 pp. 1951-1969. Techniques for Torque Controlled Light-Weight Robots”, 2002 IEEE
W. H. Zhu, S. E. Salcudean M. Zhu ; Experiments with Transparent International Conference on Robotics & Automation; Washington
Teleoperation under Position and Rate Control;.: Robotics and Auto DC May 2002.
mation, 1999. Proceedings. 1999 IEEE International Conference; Hyosig Kang and John T. Wen.: “Robotic Assistants Aid Surgeons
1870-1875 vol. 3. During Minimally Invasive Procedures' Hyosig Kang and John T.
J.H. Ryu, D.S. Kwon, B. Hannaford; Stable Teleoperation with Time Wen, IEE Magazine of Engineering in Medicine and Biology, Jan./
Domain Passivity Control "Jee-Hwan Ryu, Dong-Soo Kwon, and Feb. 2001.
 US 7.689,320 B2
Page 3

Hirche, Sandra& Buss, Martin; “Passive Position Controlled Hirche, S., and Buss, M., “Passive position controlled telepresence
Tlepresence Systems Fwith Time Delay'; American Control Confer systems with time delay.” Proceedings of the 2003 American Control
ence, 2003. Proceedings of the 2003, 2003—ieeexplore.ieee.org. Conference, 2003. vol. 1, Jun. 4-6, 2003 pp. 168-173 vol. 1.
Kuchenbecker, Katherine J & Niemeyer, G; 2004 ASME Congress Itoh, T., et al., “Human-Machine Cooperative Telemanipulation with
and RD&d Expo ; Anaheim, CA Nov. 11-19, 2004. Motion and Force Scaling Using Task-Oriented VirtualTool Dynam
Abbott, J.J., and Okamura, A.M.; “Generating a State-Space Formu ics.” IEEE Transactions on Robotics and Automation, vol. 16, Issue 5,
lation of a Class of Telemanipulator.” Haptic Exploration Lab Tech Oct. 2000 pp. 505-516.
nical Report 03-1. The Johns Hopkins University, Jun. 1, 2003, Joly, L.D., et al., “Mechanical analogies in hybrid position/force
<http://www.haptics.me.jhu.edu/publications/tech03-1-abbott. control.” Proceedings. 1997 IEEE International Conference on
pdf>. Robotics and Automation, Apr. 20-25, 1997 pp. 835-840 vol. 1.
Abbott, J.J., and Okamura, A.M.; Analysis of virtual fixture contact Kang, H., and J.T. Wen, “Robotic Assistants Aid Surgeons. During
stability for telemanipulation, Proceedings. 2003 IEEE/RSJ Interna Minimally Invasive Procedures.” IEEE Engineering in Medicine and
tional Conference on Intelligent Robots and Systems, IROS 2003, Biology Magazine, vol. 20, Issue 1, Jan.-Feb. 2001 pp. 94-104.
vol. 3, Oct. 27-31, 2003, pp. 2699-2703. Kuchenbecker, K.J., and G. Niemeyer, "Canceling Induced Master
Albu-Schaffer, A., and Hirzinger, G., “Cartesian Impedance Control Motion in Force-Reflecting Teleoperation.” Draft for IMECE Sym
Techniques for Torque Controlled Light-Weight Robots.” Proceed posium on Advances in Robot Dynamics and Control; Published in
ings. IEEE International Conference on Robotics and Automation, Proceedings, vol. 2, Paper No. 60049, Nov. 2004.
ICRA'02, vol. 1, May 11-15, 2002 pp. 657-663 vol. 1. Lawrence, D.A., “Stability and Transparency in Bilateral Teleopera
Anderson and Spong, “Bilateral Control of Teleoperators with Time tion.” IEEE Transactions on Robotics and Automation, vol. 9, Issue 5,
Delay.” IEEE Transactions on Automatic Control, vol.34, No. 5, May Oct. 1993 pp. 624-637.
1989, pp. 494-501. Mitsuishi, M., et al., “Tele-micro-Surgery system with intelligent user
Arcara, P. et al., “Intrinsically Passive Control in Bilateral Teleopera interface.” Proceedings. IEEE International Conference on Robotics
tion Mimi Systems.” Proceedings of the European Control Confer and Automation, 2000. . ICRA '00, vol. 2, Apr. 24-28, 2000 pp.
ence 2001 (ECC2001), p. 1180-1185, Porto, Portugal. 1607-1614 vol. 2.
Cavusoglu, M.C., "Control of a TelesurgicalWorkstation.” Master of Natale, C., and R. Koeppe, "An Automatic Procedure for Force Con
Science, Plan II in Department of Electrical Engineering and Com troller Design” Proceedings. 1999 IEEE/ASME International Con
puter Sciences, Univ. of California at Berkeley, 1997, 66 pages. ference on Advanced Intelligent Mechatronics, Sep. 19-23, 1999pp.
Cavusoglu, M.C., et al., "Alaparoscopic TeleSurgical Workstation.” 967-972.
IEEE Transactions on Robotics and Automation, vol. 15, No. 4. Aug. Prisco, G.M., et al., “Preliminary Considerations on the Design of
1999, pp. 728-739. Controllers for Haptic Interfaces.” SPIE Proceedings, Telemanipula
Cavusoglu, M.C., et al., “Bilateral Controller Design for tor and Telepresence Technologies IV, Ed. Stein, Matthew R., Oct.
Telemanipulation in Soft Environments.” Proceedings of the IEEE 14-17, 1997, vol. 3206, pp. 2-11.
International Conference on Robotics and Automation (ICRA 2001), Repperger, "Active Force Reflection Devices in Teleoperation.” IEEE
Seoul, Korea, May 21-26, 2001, 8 pages. Control Systems Magazine, vol. 11, Issue 1, Jan. 1991 pp. 52-56.
Cavusoglu, M.C., et al., “Design of Bilateral Teleoperation Control Rosen, J., et al., “Force Controlled and Teleoperated Endoscopic
lers for Haptic Exploration and Telemanipulation of Soft Environ Grasper for Minimally Invasive Surgery-Experimental Performance
ments.” IEEE Transactions on Robotics and Automation, vol. 18, Evaluation.” IEEE Transactions on Biomedical Engineering, vol. 46,
Issue 4. Aug. 2002 pp. 641-647.
Dubey, R.V., et al., “Variable Damping Impedance control of a Bilat Issue 10, Oct. 1999 pp. 1212-1221.
eral Telerobotic System.” IEEE Control Systems Magazine, vol. 17. Roy, J. and Whitcomb, L.L., “Adaptive Force Control of Position?
Issue 1, Feb. 1997 pp. 37-45. Velocity Controlled Robots: Theory and Experiment.” Submitted for
Flemmer, H., et al., “Passivity Issues in Bilateral Teleoperation—a Review, IEEE Transactions on Robotics and Automation, published
Phase Property.” EuroHaptics 2003, Dublin, Ireland, Jul. 6-9, 2003, in vol. 18, Issue 2, Apr. 2002 pp. 121-137.
14 pages, Internet <http://www.eurohaptics.vision.ee.ethZ.ch/2003. Roy, J., et al., “Haptic Feedback Augmentation through Position
shtml>. Based Adaptive Force Scaling: Theory and Experiments.” IEEE/RSJ
Funda, J., et al., “Constrained Cartesian Motion Control for Teleoper International Conference on Intelligent Robots and System, 2002.
ated Surgical Robots.” IEEE Transactions on Robotics and Automa vol. 3, Sep. 30–Oct. 5, 2002 pp. 2911-2919 vol. 3.
tion, (Jun. 1996) vol. 12, No. 3, pp. 453-465. Ryu, J.H., et al., "Stable Teleoperation with Time Domain Passivity
Geffard. F., et al., “On the Use of a Base Force?Torque Sensor In Control.” Internet <brl.ee.washington.edu/publications/Rep154.
Teleoperation.” Proceedings. IEEE International Conference on pdf> 6 pages, 2002.
Robotics and Automation, 2000. ICRA'00. vol. 3, Apr. 24-28, 2000 Salcudean, et al., “Performance Measurement in Scaled Teleopera
pp. 2677-2683, vol. 3. tion for MicroSurgery.” Proc. of First Joint Conf. Computer Vision,
Hacksel, P.J. and S.E. Salcudean, "Estimation of Environment Forces Virtual Reality and Robotics in Medicine and Medical Robotics and
and Rigid-Body Velocities using Observers.” Proceedings, 1994 Computer-Assisted Surgery, CVRMed-MRCAS'97, Grenoble,
IEEE International Conference on Robotics and Automation, May France, Mar. 19-22, 1997: J. TroccaZ, E. Grimson, R. Mosges (Eds.),
8-13, 1994 pp.931-936, vol. 2. pp. 789-798, Springer-Verlag Lecture Notes in Computer Science,
Hannaford, B., "A Design Framework for Teleoperators with vol. 1205.
Kinesthetic Feedback.” IEEE Transactions on Robotics and Automa Salcudean, S.E., et al., “A Robot System for Medical Ultrasound.”
tion, vol. 5, Issue 4. Aug. 1989 pp. 426-434. Robotics Research, the 9th International Symposium of Robotics
Hashtrudi-Zaad, K.H., et al., “Bilateral Parallel Force/Position Research (ISRR 99), J.M. Hollerbach and D.E. Koditschek (Eds.),
Teleoperation Control.” 1999 Symp. Haptic Interfaces for Virtual pp. 195-202, Springer 2000.
Environments and Teleoperation Systems, Intl. Mech. Eng. Congr. Salcudean, S.E., et al., “Transparent Bilateral Teleoperation under
Exp., (ASME Winter Annual Meeting), Nashville, TN, Nov. 1999, Position and Rate Control'. Intl. J. Robotics Research, vol. 19, No.
DSC Vol., pp. 351-358. 12, Dec. 2000, pp. 1185-1202.
Hashtrudi-Zaad, K.H. and Salcudean, S.E., "Analysis and Evaluation Salisbury, K., “Active stiffness control of manipulator in Cartesian
of Stability and Performance Robustness for Teleoperation Control coordinates.” In 19th IEEE Conference on Decision and Control,
Architectures.” IEEE International Conference on Robotics and 1980. pp. 95-100.
Automation, 2000. Proceedings. ICRA'00. vol. 4, Apr. 24-28, 2000 Sherman, et al., “Comparison of Teleoperator Control Architectures
pp. 3107-31 13 vol. 4. for Palpation Task.” Proceedings of IMECE'00, Symp. on Haptic
Hashtrudi-Zaad, K.H., and S.E. Salcudean, “Transparency in Time Interfaces for Virtual Environment and Teleoperator Systems, Nov.
Delayed Systems and the Effect of Local Force Feedback for Trans 5-10, 2000, Orlando, FL, 8 pages.
parent Teleoperation.” IEEE Transactions on Robotics and Automa Speich, J.E., et al., “A Method for Simultaneously Increasing Trans
tion, vol. 18, No. 1, Feb. 2002, pp. 108-114. parency and Stability Robustness in Bilateral Teleoperation.” Pro
 US 7.689,320 B2
Page 4

ceedings. IEEE International Conference on Robotics and Automa Experiments.” IEEE Transactions on Robotics and Automation, vol.
tion, 2000. ICRA'00. vol. 3, Apr. 24-28, 2000, pp. 2671-2676 vol. 3. 10, No. 5, Oct. 1994, pp. 605-620.
Stramigioli, S., “Geometric Scattering in Tele-manipulation of Port Yokokohji, Y. et al., “Bilateral Control with Energy Balance Moni
Controlled Hamiltonian Systems.” Submitted to 39th IEEE Confer toring Under Time-Varying Communication Delay.” Proceedings.
ence on Decision and Control, Dec. 12-15, 2000. Sydney, Australia. IEEE International Conference on Robotics and Automation, 2000.
Tendick, et al., “Maximizing the Sensation of Compliance in ICRA 00. vol. 3, Apr. 24-28, 2000 pp. 2684-2689 vol. 3.
Teleoperative Surgery.” Submitted to the Eighth International Sym Zhu, W.H. and S.E. Salcudean, "Teleoperation with Adaptive
posium on Robotics with Applications, part of the World Automation Motion/Force Control.” IEEE International Conference on Robotics
Congress (WAC 2000), Maui, HI, Jun. 11-16, 2000, 10 pages. and Automation (ICRA), pp. 231-237. Detroit, MI, May 1999.
Tsumaki, Y., et al., “Teleoperation Based on the Adjoint Jacobian Zhu, W.H., and S.E. Salcudean, "Stability Guaranteed Teleoperation:
Approach.” IEEE Control Systems Magazine, vol. 17. Issue 1, Feb. an Adaptive Motion/Force Control Approach.” IEEE Transactions on
1997 pp. 53-62. Automatic Control, vol. 45, Issue 11, Nov. 2000 pp. 1951-1969.
Volpe, R. and P.K. Khosla, “The Equivalence of Second-Order Zhu, W.H., et al., “Experiments with Transparent Teleoperation
Impedance Control and Proportional Gain Explicit Force Control.” under Position and Rate Control.” Proceedings of IEEE International
International Journal of Robotic Research (IJRR), vol. 14, No. 6, pp. Conference on Robotics and Automation (ICRA), 1999, vol. 3, May
574-589, Dec. 1995. 10-15, 1999 pp. 1870-1875.
Yokokohji and Yoshikawa, “Bilateral Control of Master-Slave
Manipulators for Ideal Kinesthetic Coupling- Formulation and * cited by examiner
U.S. Patent Mar. 30, 2010 Sheet 1 of 9 US 7.689,320 B2
U.S. Patent Mar. 30, 2010 Sheet 2 of 9 US 7.689,320 B2

fig.2
U.S. Patent Mar. 30, 2010 Sheet 3 of 9 US 7.689,320 B2

?ABIS ?SUÐAu])

J? SeW
U.S. Patent Mar. 30, 2010 Sheet 4 of 9 US 7.689,320 B2

“TWO ‘THW\A/
N? -0 7
TWO
U.S. Patent Mar. 30, 2010 Sheet 5 of 9 US 7.689,320 B2

TA

Y7LJNB0AIEO7SGT
U.S. Patent Mar. 30, 2010 Sheet 6 of 9 US 7.689,320 B2

["WOTIL
TWA
U.S. Patent Mar. 30, 2010 Sheet 7 of 9 US 7.689,320 B2

701

FILTER
--
DERIVATIVE
FUNCTIONS

fig 7

803

FILTER(S)
--
DERVATIVE
FUNCTIONS 401-2

fig.8
U.S. Patent Mar. 30, 2010 Sheet 8 of 9 US 7.689,320 B2

fig.10
dB
Wp Wiz Wn1 Win2
Ya
U.S. Patent Mar. 30, 2010 Sheet 9 Of 9 US 7.689,320 B2

dB

dB

fig.14
 US 7,689,320 B2
1. 2
ROBOTC SURGICAL SYSTEM WITH JOINT robotic arm assembly motion. Unfortunately, when force
MOTION CONTROLLERADAPTED TO feedback indicative of forces being asserted against the tip of
REDUCE INSTRUMENT TIP VIBRATIONS the Surgical instrument is provided back to the Surgeon
through the master manipulator, the delay added to the tele
CROSS REFERENCE TO RELATED operation control system by the low pass filter tends to drive
APPLICATIONS the control system unstable. This greatly limits either the
ability to filter out undesirable vibrations or the ability of the
This application claims priority to U.S. provisional appli system to provide the Surgeon with haptic force feedback.
cation Ser. No. 60/751,947 filed Dec. 20, 2005, which is
incorporated herein by reference. 10 OBJECTS AND SUMMARY OF THE
INVENTION
FIELD OF THE INVENTION
Accordingly, an object of aspects of the present invention is
The present invention generally relates to robotic Surgical a robotic arm assembly joint controller for reducing vibra
systems and in particular, to a robotic Surgical system with 15 tions that may otherwise be experienced on a tip of a Surgical
joint motion controller adapted to reduce Surgical instrument instrument during a Surgical procedure without adding undue
tip vibrations without significantly degrading control system delay.
stability. Another object of aspects of the present invention is a
robotic arm assembly joint controller for reducing vibrations
BACKGROUND OF THE INVENTION that may otherwise be experienced on a tip of a Surgical
instrument during a Surgical procedure, and providing force
Robotic Surgical systems such as those used in performing feedback indicative of forces being asserted against the
minimally invasive Surgical procedures offer many benefits instrument tip back to a Surgeon performing the procedure
over traditional open Surgery techniques, including less pain, without jeopardizing control system stability.
shorter hospital stays, quicker return to normal activities, 25 These and additional objects are accomplished by the vari
minimal scarring, reduced recovery time, and less injury to ous aspects of the present invention, wherein briefly stated,
tissue. Consequently, demand for minimally invasive Surgery one aspect is a robotic Surgical system comprising: a Surgical
using robotic Surgical systems is strong and growing. instrument; a robotic arm assembly holding the Surgical
One example of a robotic surgical system is the da VinciR) instrument; a master manipulator; and a controller configured
Surgical System from Intuitive Surgical, Inc., of Sunnyvale, 30 to control movement of the Surgical instrument in response to
Calif. The da VinciR) system includes a Surgeon’s console, a user operation of the master manipulator in Such a manner
patient-side cart, a high performance 3-D vision system, and that vibrations experienced at a tip of the surgical instrument
Intuitive Surgicals proprietary EndoWristTM articulating are reduced by filtering an output of the master manipulator
instruments, which are modeled after the human wrist so that that may induce the vibrations while at least partially com
when added to the motions of the robotic arm assembly hold 35 pensating in a feedback path back to the master manipulator
ing the Surgical instrument, they allow a full six degrees of for delay induced by such filtering so as to enhance stability of
freedom of motion, which is comparable to the natural Such control.
motions of open Surgery. Another aspect is a control system included in a controller
The da Vinci R. Surgeon’s console has a high-resolution for controlling movement of a robotic arm assembly in
Stereoscopic video display with two progressive scan cathode 40 response to user manipulation of a master manipulator,
ray tubes (“CRTs). The system offers higher fidelity than wherein the control system controls rotation of a slave joint of
polarization, shutter eyeglass, or other techniques. Each eye the robotic arm assembly. The control system comprises: a
views a separate CRT presenting the left or right eye perspec filter in a forward control path defined as being from the
tive, through an objective lens and a series of mirrors. The master manipulator to a motor coupled to the slave joint,
Surgeon sits comfortably and looks into this display through 45 wherein the filter attenuates frequency content of a joint posi
out Surgery, making it an ideal place for the Surgeon to display tion command generated from user manipulation of the mas
and manipulate 3-D intraoperative imagery. ter manipulator so that vibrations experienced at a tip of a
While performing a Surgical procedure, vibrations experi Surgical instrument held by the robotic arm assembly are
enced on a tip of a Surgical instrument may cause control reduced; and an inverse filter in a feedback path defined as
problems for a Surgeon. For example, such vibrations may 50 being from a slave joint sensor to the master manipulator,
make it difficult for the Surgeon to perform fine Surgical wherein the slave joint sensor senses movement of the slave
manipulations of tissue, needles and Sutures. A flexible joint and the inverse filter provides a phase lead in the feed
robotic arm assembly, especially when mounted on flexible back path so as to at least partially compensate for a phase lag
setup arms and/or with unbalanced link masses and inertias, caused by the filter.
may cause such vibrations to easily get started by fast tran 55 Another aspect is a method for controlling the movement
sient motions commanded by the Surgeon on a master of a slave joint in a robotic arm assembly, comprising: gen
manipulator. The vibrations in this case may be at the resonant erating filtered position, Velocity and acceleration slave joint
frequency of the robotic armassembly, or that of a mechanical commands so as to command movement of the slave joint
structure Supporting the robotic arm assembly. In addition, while reducing vibrations experienced on a tip of a Surgical
vibrations may also occur as a result of any non-Smooth 60 instrument held by the robotic arm assembly; generating a
command motion of the master manipulator. motor command signal using the filtered position, Velocity
Although a low pass filter may be employed in a teleop and acceleration slave joint commands; providing the motor
eration control system to filter out master manipulator motion command signal to a motor coupled to the slave joint; receiv
commands that may cause vibration occurring at frequencies ing sensed positions of the slave joint; generating estimated
above those of intended Surgical motion, Such a low pass filter 65 positions, Velocities and accelerations of the slave joint using
also inserts delay into the teleoperation system, specifically the sensed positions of the joint and the motor command
between the master commanded motion and the actual signal; and updating the motor command signal using a func
 US 7,689,320 B2
3 4
tion of the differences between the filtered position, velocity DETAILED DESCRIPTION OF THE PREFERRED
and acceleration slave joint commands and the estimated EMBODIMENT
positions, Velocities and accelerations of the slave joint.
Still another aspect is a method for reducing vibrations on FIG. 1 illustrates, as an example, a top view of an operating
a tip of a Surgical instrument held by a robotic arm assembly 5 room employing a robotic Surgical system. The robotic Sur
without degrading stability of a control system controllably gical system in this case is a robotic Surgical system 100
moving a slave joint of the robotic arm assembly. The method including a Console (“C) utilized by a Surgeon (“S”) while
comprises: attenuating a first signal indicating slave joint performing a minimally invasive diagnostic or Surgical pro
position commands generated from user manipulation of a cedure, usually with assistance from one or more Assistants
master manipulator at frequencies beyond a cut-off frequency 10 ('A'), on a Patient (“P”) who is lying down on an Operating
so that vibrations on the tip of the surgical instrument held by table (“O”).
the robotic arm assembly are reduced at those frequencies; The Console includes a monitor 104 for displaying an
and amplifying a second signal indicating a position of the image of a Surgical site to the Surgeon, left and right manipu
slave joint at frequencies above the cut-off frequency while latable control devices 108 and 109, a foot pedal 105, and a
providing phase lead to the second signal so as to at least 15 processor 102. The control devices 108 and 109 may include
partially compensate for delay caused by the attenuation of any one or more of a variety of input devices such as joysticks,
the first signal and stably generate a feedback signal provided gloves, trigger-guns, hand-operated controllers, or the like.
to the master manipulation. The processor 102 is preferably a dedicated computer that
Additional objects, features and advantages of the various may be integrated into the Console or positioned next to it.
aspects of the present invention will become apparent from The Surgeon performs a minimally invasive Surgical pro
the following description of its preferred embodiment, which cedure by manipulating the control devices 108 and 109 (also
description should be taken in conjunction with the accom referred to herein as “master manipulators’) so that the pro
panying drawings. cessor 102 causes their respectively associated robotic arm
assemblies, 128 and 129, (also referred to herein as “slave
BRIEF DESCRIPTION OF THE DRAWINGS 25 manipulators') to manipulate their respective removably
coupled surgical instruments 138 and 139 (also referred to
FIG. 1 illustrates a top view of an operating room employ herein as “tools”) accordingly, while the Surgeon views the
ing a robotic Surgical system utilizing aspects of the present surgical site in 3-D on the Console monitor 104 as it is
invention. captured by a stereoscopic endoscope 140.
FIG. 2 illustrates a simplified side view of a robotic arm 30 Each of the tools 138 and 139, as well as the endoscope
assembly that is usable with various aspects of the present 140, is preferably inserted through a cannula or other tool
invention. guide (not shown) into the Patient so as to extend down to the
FIG.3 illustrates a block diagram of a master/slave control Surgical site through a corresponding minimally invasive
system utilizing aspects of the present invention. incision such as incision 166. Each of the robotic arms is
FIG. 4 illustrates a block diagram of a slave joint controller 35 conventionally formed of links, such as link 162, which are
utilizing aspects of the present invention. coupled together and manipulated through motor controlled
FIG. 5 illustrates a block diagram of a slave joint observer or active joints, such as joint 163.
usable in a control system utilizing aspects of the present The number of Surgical tools used at one time and conse
invention. quently, the number of robotic arms being used in the system
FIG. 6 illustrates a block diagram of a master command 40 100 will generally depend on the diagnostic or Surgical pro
observer usable in a control system utilizing aspects of the cedure and the space constraints within the operating room,
present invention. among other factors. If it is necessary to change one or more
FIG. 7 illustrates a block diagram of a first filter/derivative of the tools being used during a procedure, the Assistant may
function usable in a control system utilizing aspects of the remove the tool no longer being used from its robotic arm, and
present invention. 45 replace it with another tool 131 from a Tray (“T”) in the
FIG. 8 illustrates a block diagram of a second filter/deriva operating room.
tive function usable in a control system utilizing aspects of the Preferably, the monitor 104 is positioned near the Sur
present invention. geon’s hands so that it will display a projected image that is
FIG. 9 illustrates a frequency diagram for a fixed attenua oriented so that the Surgeon feels that he or she is actually
tion filter usable in a control system utilizing aspects of the 50 looking directly down onto the operating site. To that end,
present invention. images of the tools 138 and 139 preferably appear to be
FIG. 10 illustrates a frequency diagram for a dual fre located substantially where the Surgeons hands are located.
quency notch filterusable in a control system utilizing aspects The processor 102 performs various functions in the sys
of the present invention. tem 100. One important function that it performs is to trans
55 late and transfer the mechanical motion of control devices
FIG. 11 illustrates a frequency diagram for a combination 108 and 109 to their respective robotic arms 128 and 129
filter usable in a control system utilizing aspects of the present through control signals over bus 110 So that the Surgeon can
invention. effectively manipulate their respective tools 138 and 139.
FIG. 12 illustrates a frequency diagram for an inverse fixed Another important function is to implement various control
attenuation filter usable in a control system utilizing aspects 60 system processes as described herein.
of the present invention. Although described as a processor, it is to be appreciated
FIG. 13 illustrates a frequency diagram for an inverse dual that the processor 102 may be implemented in practice by any
frequency notch filter usable in a control system utilizing combination of hardware, software and firmware. Also, its
aspects of the present invention. functions as described herein may be performed by one unit,
FIG. 14 illustrates a frequency diagram for an inverse com 65 or divided up among different components, each of which
bination filter usable in a control system utilizing aspects of may be implemented in turn by any combination of hardware,
the present invention. software and firmware.
 US 7,689,320 B2
5 6
For additional details on robotic Surgical systems, see, e.g., the slave manipulator 128 and consequently, the position and
commonly owned U.S. Pat. No. 6,493,608 Aspects of a orientation of its attached tool 138, as commanded by move
Control System of a Minimally Invasive Surgical Apparatus.” ment of the master manipulator 108 by a surgeon. A similar
and commonly owned U.S. Pat. No. 6,671.581 “Camera Ref control system may also be provided for slave manipulator
erenced Control in a Minimally Invasive Surgical Apparatus.” 129 and its associated master manipulator 109.
which are incorporated herein by these references. Both the master and slave manipulators, 108 and 128,
FIG. 2 illustrates, as an example, a side view of a simplified include a number of linkages connected by joints so as to
(not necessarily in proportion or complete) robotic arm facilitate multiple degrees-of-freedom movement. As the Sur
assembly 200 (which is representative of robotic arm assem geon moves the master manipulator 108 from one position to
blies 128 and 129) holding a surgical instrument 250 (which 10 another during the course of performing a Surgical procedure,
is representative of tools 138 and 139) for performing a sur sensors associated with the master manipulator joints provide
gical procedure. The surgical instrument 250 is removably information indicating Such command movement in master
held in tool holder 240. The arm assembly 200 is mechani joint space, and sensors associated with the slave manipulator
cally supported by a base 201, which may be part of a patient joints provide information indicating slave manipulator 128
side movable cart or affixed to the operating table or ceiling. 15 and consequently, tool 138 movement in slave joint space for
It includes links 202 and 203 which are coupled together and feedback purposes.
to the base 201 through setup joints 204 and 205. A master input processing unit 301 receives the informa
The setup joints 204 and 205 in this example are passive tion of the master joint positions, which are sampled at the
joints that allow manual positioning of the arm 200 when their control system processing rate (e.g., 1300 Hz in the present
brakes are released. For example, setup joint 204 allows link example), from the masterjoint sensors in the master manipu
202 to be manually rotated about axis 206, and setup joint 205 lator 108, and computes joint velocities from the sensed joint
allows link 203 to be manually rotated about axis 207. positions. A master forward kinematics processing unit 302
Although only two links and two setup joints are shown in receives the master joint positions and Velocities from the
this example, more or less of each may be used as appropriate master input processing unit 301, transforms them from mas
in this and other robotic arm assemblies in conjunction with 25 terjoint space to corresponding positions and Velocities of the
the present invention. For example, although setup joints 204 master frame (i.e., the frame associated with the master
and 205 are useful for horizontal positioning of the arm 200, manipulator 108) in Cartesian space relative to the eye refer
additional setup joints may be included and useful for limited ence frame (i.e., the reference frame associated with the posi
Vertical and angular positioning of the arm 200. For major tion of the Surgeon’s eyes), using, for example, a Jacobian
vertical positioning of the arm 200, however, the arm 200 may 30 matrix and eye related information separately determined and
also be slidably moved along the vertical axis of the base 201 provided in block 303.
and locked in position. A scale and offset processing unit 304 receives the Carte
The robotic arm assembly 200 also includes three active sian position and Velocity commands from the master forward
joints driven by motors. A yaw joint 210 allows arm section kinematics processing unit 302, Scales the commanded move
230 to rotate around an axis 261, and a pitch joint 220 allows 35 ment according to a scale factor selected to perform the Sur
arm section 230 to rotate about an axis perpendicular to that of gical procedure, and takes into account offsets to generate
axis 261 and orthogonal to the plane of the drawing. The arm desired slave tool frame (i.e., the frame associated with the
section 230 is configured so that sections 231 and 232 are tool 138) positions and velocities. The scale adjustment is
always parallel to each other as the pitch joint 220 is rotated useful where small movements of the slave manipulator 128
by its motor. As a consequence, the instrument 250 may be 40 are desired relative to larger movement of the master manipu
controllably moved by driving the yaw and pitch motors so as lator 108 in order to allow more precise movement of the slave
to pivot about the pivot point 262, which is generally located tool 138 at the surgical site. The offsets, on the other hand,
through manual positioning of the setup joints 204 and 205 so determine, for example, the corresponding position and/or
as to be at the point of incision into the patient. In addition, an orientation of an end effector frame (e.g., the frame associ
insertion gear 245 may be coupled to a linear drive mecha 45 ated with an end effector of the tool 138) in the camera
nism (not shown) to extend or retract the instrument 250 reference frame (i.e., the frame associated with the distal tip
along its axis 263. of the endoscope 140) relative to a position and orientation of
As noted, one problem with the robotic arm assembly 200 the master frame in the eye reference frame.
is the occurrence of vibrations on a tip 251 of the surgical A simulated slave processing unit 308 receives desired
instrument 250 that may cause control problems for a surgeon 50 slave tool frame position and Velocity commands from the
when performing a Surgical procedure using the instrument scale and offset processing unit 304, and limits the desired
250. The vibrations may be triggered by dynamic reaction slave tool frame position, orientation and Velocities, to
forces produced by the robotic arm assembly 200 during the assigned Cartesian Limits for instance to enforce correct and
motions of the surgical instrument 250. The vibrations may be intuitive operation of the tool 138 by keeping it within its
amplified by resonance of the robotic arm assembly 200 or a 55 dexterous workspace. The simulated slave processing unit
mechanical structure Supporting the robotic arm assembly 308 generates simulated slave joint positions and velocities
200, such as the base 201, at their respective resonant fre corresponding to the limited slave tool frame positions and
quencies. Velocities, while making Sure that the generated slave joint
Although each of the yaw, pitch and insertion joints or positions and Velocities do not exceed the actual slave joints
gears, 210, 220 and 245, is controlled by an individual joint or 60 range of motion and maximum velocities (i.e., joint limits)
gear controller, the three controllers are controlled by a com even in the vicinity of kinematic singularities for the slave
mon master/slave control system so that the robotic arm kinematics.
assembly 200 (also referred to herein as a “slave manipula An inverse scale and offset processing unit 306 receives the
tor) may be controlled through user (e.g., Surgeon) manipu simulated joint position and Velocity commands from the
lation of its associated master manipulator. 65 simulated slave processing unit 308, and performs an inverse
FIG. 3 illustrates, as an example, a block diagram of a function to that of the scale and offset processing unit 304 on
master/slave control system 300 for controlling movement of them. A Cartesian controller 307 receives as first inputs, the
 US 7,689,320 B2
7 8
inputs to the scale and offset processing unit 304 and as along with the Cartesian force provided by the Cartesian
second inputs, the outputs of the inverse scale and offset controller 307 through the Master transpose kinematics pro
processing unit 306. The Cartesian controller 307 then gen cessing unit 315 and Master output processing 316 as previ
erates an error signal as a difference of the first and second ously described in reference to their processing of the Carte
inputs, and a Cartesian force “F, from the error signal sian force provided by the Cartesian controller 307.
such as with the following formula: Additional details related to conventional aspects of the
F-K(Ax)+B (Ak) (1) master/slave control system 300, such as the various refer
ence frames referred to herein and the calculation of the
where “K” is a spring constant, “B” is a damping constant, surgeon eye related information provided in block 303 and
“AX' is the difference between the Cartesian velocity inputs 10 the slave fulcrum information provided in block 312, which
to the Cartesian controller 307 and “AX is the difference are based upon well-known mathematics, are described, for
between the Cartesian position inputs to the Cartesian con example, in commonly owned U.S. Pat. No. 6,424.885,
troller 307. Foran orientation error, a corresponding torque in “Camera Referenced Control in a Minimally Invasive Surgi
Cartesian space is determined. cal Apparatus, which is incorporated herein by this refer
A master transpose kinematics processing unit 315 15 CCC.
receives the Cartesian force F through a Summation node The joint control unit 400 includes a joint controller for
314, and generates a corresponding torque in joint space each slave joint of the slave manipulator 128 that is being
using, for example, the Jacobian transpose matrix and kine controlled by the master/slave control system 300. In particu
matic relationships associated with the master manipulator lar, where the slave manipulator 128 includes a yaw joint 210,
108. A master output processing unit 316 receives the master a pitch joint 220, and an insertion axis gear 245. Such as the
torque signals from the master transpose kinematics process robotic arm assembly 200 of FIG. 2, each of these joints or
ing unit 315, generates electrical currents corresponding to gears will have its own controller.
the master torque signals, and Supplies the electrical currents FIG. 4 illustrates, as an example, a block diagram of one
to corresponding master joint motors of the master manipu such joint controller 400-1 (e.g., for controlling movement of
lator 108. As a result, a Surgeon operating the master manipu 25 either the yaw, pitch, or insertion joint or gear, 210, 220 or
lator 108 feels the Cartesian force, F, whenever the sur 245, of the robotic arm assembly 200), which has been
geon is commanding a position or Velocity which exceeds adapted to reduce vibrations on the Surgical instrument or end
system Cartesian or slave joint limits, or would result in a effector tip without overly degrading the stability of the con
kinematic singularity condition for the slave manipulator trol system 400 with force feedback. To simplify the descrip
128. 30 tion herein and in the claims, the term “joint' is to be under
As the master input processing unit 301 is receiving master stood to include gears as well as any other controllable
joint positions from sensors in the master manipulator 108, a component coupled to linear drive mechanisms that may be
slave input processing unit 309 is also receiving slave joint used in controlling robotic arm assemblies.
positions from position sensors in the slave manipulator 128 First, to reduce the vibrations at the tip of the instrument, an
at the control system processing rate. A joint control unit 400 35 appropriate filter (plus derivative functions S and S) 401 is
receives the slave joint positions from the slave input process introduced into a forward path of the joint controller 400-1.
ing unit 309 and the simulated joint position commands pro The derivative functions are included, because the joint con
vided from the simulated slave processing unit 308, and gen troller 400-1 employs a slave feedforward (“FFD) and feed
erates slave torque command signals for the slavejoint motors back (“FBK') controller 402, which requires joint velocity
and master torque feedback command signals for the master 40 and acceleration commands, as well as a joint position com
joint motors. mand. The controller 402 can be implemented as a propor
The slave torque command signals are generated by the tional-integral-derivative (“PID) controller, a non-linear
joint control unit 400 so as to drive joints of the slave manipu PID controller, or other controller including those having
lator 128 until feedback errors calculated in the joint control acceleration feedback and feed forward of the joint dynamics.
unit 400 zero out, as will be described in further detail in 45 The filter used in block 401 (e.g., 701 in FIG. 7 and 803, 804
reference to FIG. 4. A slave output processing unit 310 and 805 in FIG. 8) is preferably a linear filter. The derivatives
receives the slave torque command signals from the joint of a joint position command QMJ provided to the block 401
control unit 400, converts them into appropriate electrical may be calculated after filtering of the joint position com
currents, and Supplies the electrical currents to the joint mand QMJ as shown in FIG. 7, or they may be calculated
motors of the slave manipulator 128 so as to drive the motors 50 before filtering of the joint position command QMJ as shown
accordingly. in FIG.8. In the latter case, each of the joint position, Velocity
The master torque feedback command signals are gener and acceleration commands may be individually filtered
ated by the joint control unit 400 as a function of the slave using the same or different filters, 803-805. Preferably, the
joint position and Velocity tracking errors so as to reflect block 401 (i.e., the filter+derivative functions) is a digital state
forces being exerted against the tool 138 or its slave manipu 55 space balanced realization using standard techniques for its
lator 128 back to the master manipulator 108 so that they may conversion from the continuous domain to the digital domain
be felt by the Surgeon. A kinematic mapping unit 311 receives (e.g., Tustin method, Euler method, etc.). The joint position
the master torque feedback command signals from the joint command QMJ is one of the simulated joint position com
control unit 400, and generates the corresponding Cartesian mands from the simulated slave processing unit 308. The
force at the tip of the tool 138 relative to the camera frame of 60 filtered joint position, Velocity and acceleration commands
the endoscope 140 using the slave kinematic configuration QC, VC and AC are then provided to the slave controller 402.
and the previously calculated slave fulcrum (e.g., pivot point) The filter (e.g., 701 in FIG. 7 or 803-805 in FIG. 8) is
position information provided in block 312. configured to attenuate frequencies beyond those reasonably
Again 313 adjusts the magnitude of the Cartesian force so expected to be commanded by user manipulation of the mas
as to ensure system stability while providing adequate force 65 termanipulator 108. Since such intended surgical motion is
sensation to the Surgeon. The gain adjusted Cartesian force is generally found to be in the 0 to 3 Hz range, a 3 Hz cut-off
then passed through the Summation node 314, and processed frequency for the filter may be appropriate in a typical case.
 US 7,689,320 B2
9 10
To ensure a reasonably sharp cut-off, multiple poles at the of the joint observer 404 is shown configured as a disturbance
cut-off frequency may be desired, such as two poles. observer structure in FIG. 5, wherein the parameters B1 and
Since the filter in block 401 inserts delay into the joint M1 come from a rigid model of the slave joint dynamics 403,
controller 400-1, an inverse filter 405 is inserted in a feedback and the parameters L1, L2 and L3 are selected empirically
path to the master manipulator 108 to at least partially com 5 through simulation and/or system calibration to provide good
pensate for that delay. The inverse filter 405 is also preferably observer dynamics and accuracy. Note that the outputs of the
implemented in state space. In order to be the inverse of the joint observer 404 are provided to the inverse filter 405 and
forward path filter (e.g., 701 in FIG. 7 or 803-805 in FIG. 8), the slave controller 402 so that differences between corre
it is configured so as to have a reciprocal transfer function. sponding of the joint position QC, Velocity VC and accelera
Thus, it is characterized as amplifying its incoming signal 10 tion AC commands, and estimated joint position QL, Velocity
starting at the same cut-off frequency as the forward path filter VL and acceleration AL may be computed by the slave con
So as to add phase lead instead of lag. troller 402 in generating the torque command signal TSJ.
To make the inverse filter 405 realizable, the forward path Also, to further enhance the accuracy and stability of the
filter is designed so that it does not attenuate continuously joint controller 400-1 and/or master/slave control system 300,
after the cut-off frequency. Instead, it is preferably configured 15 a command observer 406 is preferably inserted so as to esti
to attenuate at a constant level after reaching a predetermined mate joint position QML, Velocity VML and acceleration
attenuation level such as 12 dB. AML commands using the joint position command QMJ and
In a preferred embodiment of the inverse filter 405, the a torque feedback command TMJ. An example of the com
inverse filter is configured to have an equal number of poles mand observer 406 is shown configured as a disturbance
and Zeroes. Further, it has been found in certain applications observer structure in FIG. 6, wherein the parameters B2 and
that a forward path filter (e.g., 701 or 803-805) having two M2 come from a rigid model of the master dynamics, and the
real poles and a pair of complex Zeroes, provides good char parameters L1, L2 and L3 are selected empirically through
acteristics and a realizable and workable inverse filter 405. In simulation and/or system calibration to provide good
this configuration, the complex Zeroes define a frequency observer dynamics and accuracy (and may not necessarily be
greater than the cut-off frequency defined by the real poles so 25 the same values as their counterparts in FIG. 5 even though
that following that frequency, the attenuation remains con they have the same parameter names). Note that the outputs of
stant. FIGS. 9 and 11 respectively illustrate examples for a the command observer 406 are provided as negative values to
forward path filter and inverse filter with the characteristics the master controller 407 along with the outputs of the inverse
described thus far as having real poles at the frequency Wp filter 405, so that differences between corresponding of the
and complex pair of Zeroes at the frequency WZ, with all 30 inverse filtered, estimated joint positions QL, Velocities VL
frequencies in radians. and accelerations AL, and estimated joint position QML,
In addition to having frequency characteristics such as velocity VML and acceleration AML commands may be cal
shown in FIG. 9, the forward path filter also preferably culated and used by the master controller 407 in generating
includes notch filter characteristics such as shown in FIG. 10 the feedback torque TMJ. The master controller 407 in this
to reduce the effects of resonant frequencies, Wn1 and Win2, 35 case is preferably identical in construction as the slave con
respectively of the robotic arm assembly and a mechanical troller 402.
structure Supporting the robotic arm assembly. Consequently, Both the observers 404 and 406 may be based on joint
when combining the filtering characteristics of FIGS. 9 and models including one or more resonances. Additional details
10, the filtering characteristics as shown in FIG. 11 are those on Such observers are described, for example, in commonly
desired for the forward path filter, and in a similar fashion, the 40 owned U.S. application Ser. No. 1 1/479,144“Control System
desired filtering characteristics of the inverse filter 405 are for Reducing Internally Generated Frictional and Inertial
shown in FIG. 14. Resistance to Manual Positioning of a Surgical Manipulator.”
The forward path filter may be represented in state space as filed Jun. 30, 2006, which is incorporated herein by this
the following: reference.
45 Although the various aspects of the present invention have
been described with respect to a preferred embodiment, it will
be understood that the invention is entitled to full protection
within the full scope of the appended claims.
wherein the input u joint position command determined
using inverse kinematics; the outputy-filtered joint position, 50 We claim:
Velocity and acceleration commands; the state X is preferably 1. A robotic Surgical system comprising:
a vector having a length up to 7 by 1; and the matrices A, B, a Surgical instrument;
C and Dare determined so as to provide the desired frequency a robotic arm assembly holding the Surgical instrument;
characteristics, while having a balanced realization which a master manipulator, and
improves numerical stability in floating point. The inverse 55 a controller configured to control movement of the Surgical
filter 405 may be represented in state space form in a similar instrument in response to user operation of the master
fashion. manipulator in Such a manner that vibrations experi
To further enhance the accuracy (by reducing noise) and enced at a tip of the Surgical instrument are reduced by
stability (by reducing loop delays) of the joint controller filtering an output of the master manipulator that may
400-1 and/or master/slave control system 300, a joint 60 induce the vibrations while at least partially compensat
observer 404 is preferably inserted so as to estimate actual ing in a feedback path to the master manipulator for
joint positions, Velocities and accelerations, QL, VL and AL, delay induced by Such filtering so as to enhance stability
using sensed position indications QJ from an encoder or of Such control.
sensor coupled to the slave joint, and torque commands TSJ 2. The robotic Surgical system according to claim 1,
provided to drive the slave joint motor. The slave joint motor, 65 wherein the controller includes a control system to control
slave joint, and slave joint sensor are all included and indi movement of the robotic arm assembly in response to the user
cated in this example by reference number 403. An example operation of the master manipulator, wherein the control sys
 US 7,689,320 B2
11 12
tem includes a filter for filtering the vibrations, and an inverse observer having: inputs including a sensed slave joint posi
filter to at least partially compensate for delay caused by the tion of a slave joint in the robotic arm assembly, and a motor
filter in the control system. drive command for driving a slave joint motor adapted to
3. The robotic Surgical system according to claim 2, move the slave joint; and outputs including at least estimated
wherein the filter is characterized by attenuating high fre values for a position of the slave joint and a velocity of the
quency signals above a cut-off frequency more than low fre slave joint.
quency signals below the cut-off frequency. 16. The robotic surgical system according to claim 15,
4. The robotic Surgical system according to claim 2, wherein the outputs of the slave joint observer further
wherein the filter is further characterized by attenuating a includes an estimated value for an acceleration of the slave
resonant frequency signal caused by resonance of a mechani 10 joint.
cal structure Supporting the robotic arm assembly so as to 17. The robotic surgical system according to claim 15,
resemble a notch filter. wherein the slave joint observer has a disturbance observer
5. The robotic Surgical system according to claim 2, structure and the outputs of the slave joint observer further
wherein the filter is further characterized by attenuating a includes an estimated value for an external load being applied
resonant frequency signal caused by resonance of the robotic 15 to the slave joint motor.
arm assembly so as to resemble a notch filter. 18. The robotic surgical system according to claim 15,
6. The robotic Surgical system according to claim 2, wherein the motor drive command is generated by using at
wherein the filter is implemented in a forward path of the least one of the differences between corresponding of the
control system from the master manipulator to the robotic filtered values for the commanded joint position and com
arm assembly and the inverse filter is implemented in a feed manded joint Velocity, and estimated values for the slave joint
back path of the control system from the robotic arm assem position and Velocity.
bly to the master manipulator. 19. The robotic surgical system according to claim 15,
7. The robotic Surgical system according to claim 2, wherein part of the motor drive command is generated by
wherein the filter is implemented such that the filter has an using at least one of the filtered values for the commanded
equal number of dominant poles and Zeroes. 25 position, Velocity, and acceleration.
8. The robotic Surgical system according to claim 2, 20. The robotic Surgical system according to claim 2, fur
wherein the filter is implemented such that the dominant ther comprising a master command observer having: inputs
poles of the filter are real and the dominant Zeroes of the filter including a commanded joint position and a master motor
are complex. drive feedback value; and outputs including estimated values
9. The robotic Surgical system according to claim 2, 30 for the commanded position and Velocity.
wherein the filter is implemented such that the dominant 21. The robotic Surgical system according to claim 20,
poles of the filter are less in magnitude than a resonant fre wherein the master command observer outputs further
quency of a mechanical structure associated with the robotic including an estimated value for the command acceleration.
arm assembly. 22. The robotic Surgical system according to claim 21,
10. The robotic Surgical system according to claim 2, 35 wherein the master command observer has a disturbance
wherein an input of the filter includes at least one of a com observer structure and the outputs of the master command
manded position, commanded Velocity, and commanded observer further includes an estimated value for an external
acceleration. load applied to the master manipulator.
11. The robotic Surgical system according to claim 10, 23. The robotic Surgical system according to claim 2,
40 wherein the inverse filter receives at least one of an estimated
wherein an output of the filter includes a filtered value for the slave joint position, Velocity and acceleration, and generates
commanded position, and a filtered value for the commanded a filtered value for the at least one of the estimated slave
velocity.
12. The robotic Surgical system according to claim 11, position, Velocity and acceleration.
wherein the output of the filter further includes a filtered value 24. The robotic Surgical system according to claim 23,
for the commanded acceleration. 45 wherein the master motor drive feedback value is generated
13. The robotic Surgical system according to claim 11, by using at least one of the differences between correspond
wherein the filter is digitally implemented as a state space ing of the estimated values for the commanded position,
balanced realization of the filter. Velocity and acceleration, and outputs of the inverse filter.
14. The robotic Surgical system according to claim 11, 25. The robotic Surgical system according to claim 24,
wherein the commanded position is a position of a joint in the 50 wherein part of the master motor drive command is generated
robotic arm assembly. by using at least one of the filtered values for the commanded
15. The robotic Surgical system according to claim 2, position, Velocity and acceleration.
wherein the control system further comprises a slave joint k k k k k