Sensitivity Analysis of Performance Tests for Six-Degree-of-Freedom Serial Industrial Robots

In recent years, industrial robots have been widely used in industries for their versatility, high efficiency, and high reliability. The International Federation of Robotics has released the World Robotics 2021 Industrial Robots report, which shows that 3 million industrial robots are operating in factories worldwide in 2021. The global industrial robot shipments reached 380,000 units in 2020 [1]. With the rapid development of industrial robots, performance tests of industrial robots are also becoming more and more important. Some institutions and organizations have developed performance test standards for industrial robots. The current international standard for industrial robots is “ISO 9283:1998 Manipulating industrial robots-Performance criteria and related test methods” [2], published by the International Standards Organization (ISO), which is one of the widely used standards for industrial robots. The Association of German Engineers (VDI) and the American National Standards Institute (ANSI) also published their standards for performance tests of industrial robots: VDI 2861:1988 and ANSI/RIA R15.05 [3], respectively. Based on ISO 9283:1998, the Standardization Administration of China has also developed the GB/T 12642-2013 national standards for performance evaluation of industrial robots.

For industrial robots, pose accuracy or repeatability is one of the most important performance tests. In the ISO 9283:1998 Standard, pose accuracy (respectively repeatability) is divided into positioning accuracy (respectively repeatability) and orientation accuracy (respectively repeatability). These four tests are marked as positioning/orientation accuracy/repeatability (or pose accuracy/repeatability) in this paper. Based on the study of Refs. [4–7], there are different factors that lead to positioning/orientation errors of industrial robots, which can basically be classified into geometric and non-geometric errors. Geometric errors, also known as kinematic errors, include the errors from the manufacturing and assembly of industrial robots [8], such as inaccurate linkage lengths and misalignment between theoretical and actual axes of joints. Non-geometric errors mainly include flexible deformation of joints and links due to end loads, thermal deformation of joint/link parts caused by temperature fluctuations, gear clearance errors of reducers, etc. Judd [9] concluded that almost 90% of industrial robot positioning/orientation errors are incurred by geometric errors. However, the effect of non-geometric errors cannot be still ignored in real cases where high robot accuracy is required.

There are some existing papers about performance tests of industrial robots in the literature. Brinker et al. [10] analyzed the motion performance evaluation of parallel robots, and designed performance evaluation indexes for them. Barnfather et al. [11] designed the test target and developed a method to evaluate the machining performance of industrial robots. Rossmeissl et al. [12] analyzed the applicability of existing evaluation criteria to lightweight industrial robots, and they qualitatively proposed 13 evaluation criteria such as generality, mobility, and load capacity. Bi et al. [13] analyzed recent advances in standardized tests of robot systems and components, and described the limitations of existing industrial robot standards. Finally, they summarized the trends in the standardization of industrial robot technologies. Lin et al. [14] optimized the machining poses of a 6R industrial robot based on performance evaluation indexes to improve its motion and stiffness performance. Although the research papers and the international standards above have discussed various performance evaluation methods for industrial robots, there is still no discussion on the sensitivity of performance evaluation methods to positioning/orientation error sources. That is to say, it is still unknown whether the existing test indexes of ISO 9283:1998 are sensitive to error sources leading to positioning/orientation errors of industrial robots, including geometric and non-geometric error sources.

As addressed above, the main positioning/orientation errors of industrial robots are geometric errors, and the problem of stiffness still cannot be ignored for those industrial robots with requirements of high precision and large loads. At present, end poses of industrial robots are controlled mainly based on the classical kinematic model (i.e., the D–H model). Furthermore, there are joint stiffness compensation algorithms in many control systems of six-DOF (degree-of-freedom) serial industrial robots. As a result, parameter errors of kinematic models and stiffness models of industrial robots in their control systems are the main sources of robot end positioning/orientation errors.

A kinematic model of an industrial robot is a mathematical description between the geometric parameters and its end pose. The traditional kinematic model of industrial robots is the D–H model, which links the joint coordinate systems between adjacent links of an industrial robot through several coordinate transformation matrices. Each coordinate transformation matrix has two translation parameters and two rotation parameters (see the description in Sec. 3.1). Kinematic modeling of industrial robots has been a hotspot in the robotics community. In order to improve the positioning/orientation accuracy of a six-DOF industrial robot, He et al. [15] calibrate the robot kinematic model parameters by controlling the robot to reach the same position in different poses and using multiple position constraints. Gao et al. [16] proposed a parameter identification method of the D–H model for industrial robots. In their method, the kinematic model was linearized to obtain the parameter identification coefficient matrix, and the singular value decomposition was used to identify the redundant parameters and remove them from the matrix. Finally, the redundant parameters of the robot were estimated using an improved least-squares method. Peng et al. [17] proposed an improved calibration method based on a short chain in order to identify the open-loop and closed-loop geometric parameter errors of the D–H model. Although there has been extensive research on the D–H model, there are no reports on the relationship between D–H model parameter errors and performance tests of industrial robots.

Non-geometric errors include joint flexibility errors, gear clearance errors, errors caused by temperature changes, etc. [18–20]. The main non-geometric errors are the flexible error caused by link weight and end loads. The six-DOF serial industrial robot structure is a kinematic chain consisting of rigid or flexible links connected to each other by joints. Stiffness is defined as the ability of a mechanical system to withstand loads without large changes in its geometry, which is referred to as deformation or flexible displacement [21]. The existing analyzing methods of joint stiffness can be divided into three categories: finite element analysis, matrix structure analysis, and virtual joint modeling (VJM) [22]. The VJM method uses the coupling of rigid links and virtual elastic joints to model the stiffness of industrial robots through Jacobi mapping. The VJM method is currently popular for robot stiffness analysis and modeling. Cvitanic et al. [23] used static and dynamic stiffness models to compare and optimize robot poses under different machining conditions. Dumas et al. [24] identified the joint stiffness of serial industrial robots to evaluate the translational and rotational displacements of the robot end effector under external loads. They also analyzed the robustness of the identification method and the resulting sensitivity to measurement errors and the experiment number. Bu et al. [25] proposed a Cartesian flexibility model to describe the robot stiffness in the Cartesian space. Based on this flexible model, a quantitative evaluation index of robot machining performance was defined, and the performance index of drilling tool orientation was optimized by selecting an appropriate drilling pose. Li et al. [26] proposed a serial robot joint stiffness identification algorithm and a deformation compensation algorithm based on the dual quaternion algebra. Although there has also been extensive research on stiffness models, there are no reports on the relationship between stiffness model parameter errors and performance tests of industrial robots.

Without doubt, it is desirable that the tests described in the ISO 9283:1998 Standard [2] should be sensitive to those geometric and non-geometric error sources. Thus, any resulting positioning/orientation errors can be identified during performance evaluation of industrial robots. The sensitivity analysis method based on numerical simulation and sensitivity matrices is an effective approach to validate performance tests of dimensional measurement instruments or equipment. After addressing the different geometric misalignments that lead to systematic errors of laser trackers, Muralikrishnan et al. [27] presented the sensitivity analysis results of the performance tests in the ASME B89.4.19 Standard. Then, they proposed new length measurement system tests that demonstrate improved sensitivity to some misalignments. They also focused on the sensitivity to different error sources for performance evaluation methods of X-ray computed tomography instruments [28]. The sensitivity analysis provides a quantitative scheme for the effectiveness study of the performance evaluation methods of dimensional measuring equipment. Research on the sensitivity of industrial robot performance evaluation methods to its end positioning/orientation error sources has not been reported. Inspired by the research presented by Muralikrishnan et al. [27,28] for the measurement instruments, this study aims to analyze and improve performance evaluation methods for industrial robots based on the sensitivity analysis method. The procedure of the research in this study is illustrated in Fig. 1.

In order to analyze whether the existing tests of ISO 9283:1998 are sensitive to the parameter errors of the kinematic and the joint stiffness models leading to positioning/orientation errors of industrial robots, a sensitivity simulation method for industrial robot performance tests is developed in this study. Based on the sensitivities analysis result, the improved performance tests of industrial robots are proposed and verified by the experiment. This study may be beneficial to improvements of the performance test standards of industrial robots.

The paper is organized as follows. After introducing the performance test methods and the test indexes of industrial robots based on the ISO 9283:1998 Standard in Sec. 1, the kinematic model and the joint stiffness model of six-DOF serial industrial robots are described in Sec. 2. Then we analyze the sensitivity of positioning/orientation accuracy/repeatability of industrial robots to model parameter errors by simulation, as well as the spatial distribution characteristics of the key position index sensitivities in Secs. 3 and 4. Finally, the experiment verification and the conclusions are given. This study may be beneficial for improving industrial robot performance testing standards and the product quality of industrial robot manufacturers.

The popular international standard for industrial robot performance tests is “ISO 9283:1998 manipulating industrial robots-performance criteria and related test methods” [2]. This standard defines 14 industrial robot performance indexes and their test methods, including: pose accuracy, pose repeatability, multi-directional pose accuracy variation, distance accuracy, distance repeatability, etc. [2]. The pose accuracy (respectively repeatability) in this standard is further divided into the positioning accuracy (respectively repeatability) and the orientation accuracy (respectively repeatability), which are most commonly used by robot manufacturers and users. The tests described in the standard are mainly used to analyze and check the performance of industrial robots, and they can also be used for their prototype tests and acceptance tests. Meanwhile, ISO 9283:1998 specifies [2] that all tested poses and trajectories in the robot performance tests should be in the same workspace, which is defined as a cube. It suggests that the cube should be selected as the test portion in the robot working space. Additionally, this cube should be set to meet the following requirements: the cube is located in the part of the workspace where the most applications of the robot are expected; the cube has the largest volume with its prismatic edges parallel to the robot base coordinate system. As shown in Fig. 2, the test poses are located at five measurement points P1–P5 on the diagonal of the measurement plane. The points P1–P5 are the robot wrist reference point locations, and the final measured point of each test pose is the location of the tool center point (TCP) with axial and radial offsets.

Besides positioning/orientation accuracy/repeatability (i.e., pose accuracy and pose repeatability), there are totally 14 tests in the ISO 9283:1998 Standard including distance accuracy and repeatability, multi-directional pose accuracy variation, position stabilization time, path accuracy and path repeatability, etc. In this study, only positioning/orientation accuracy/repeatability is considered due to space limitations.

In the kinematic or stiffness modeling of industrial robots, besides using a 3 × 3 rotation matrix to describe the object orientation, the rotation matrix can also be represented by three independent parameters.

Different error sources may have different effects on the end positioning/orientation accuracy/repeatability of industrial robots, and abilities of different robot performance tests to reflect the error sources are also different. It is desirable that the tests described in the ISO 9283:1998 Standard [2] should be sensitive to those geometric and non-geometric error sources affecting the end positioning/orientation accuracy/repeatability. As introduced above, parameter errors of kinematic models and stiffness models of industrial robots in their control systems are the main error sources of robot positioning/orientation errors. In order to investigate the effects of different error sources on different robot performance tests in the ISO 9283:1998 Standard, the kinematic and the joint stiffness models for six-DOF serial industrial robots are described in this section.

Robot kinematics describes the relationships of poses, velocities, and accelerations among the various links of industrial robots. Most traditional serial industrial robots are open kinematic chains consisting of several links connected in series by rotating joints. Servo motors drive the joints, and each link can rotate around the axis of its revolute joint.

The Denavit–Hartenberg (D–H) parameters are the four parameters being related to a particular convention for attaching reference frames to the links of a spatial kinematic chain, or robot manipulator. The D–H model creates a coordinate system on each link of an industrial robot by analyzing the robot joint structure, and describes the position and orientation relationships between adjacent links through a 4 × 4 homogeneous transformation matrix. Usually, the four geometric parameters involved in the coordinate system transformation correspond between link i − 1 and link i, including the link length ai, the twist angle αi, the joint offset di, and the joint angle θi, where i = 1, 2, …, 6 for an usual six-DOF serial industrial robot. Then the coordinate transformation matrix of each joint between these two links could be constructed. By multiplying the six link transformation matrices, the transformation matrix of the industrial robot from the sixth joint coordinate system to the base coordinate system can be obtained. That is to say, the position and orientation of the robot end effector in the base coordinate system can be determined by the 24 parameters of the D–H model, i.e., (ai, αi, di, θi), where i = 1, 2, …, 6. Although the 24 parameters are often identified and used as the kinematic model parameters to control the robot pose, they cannot be identified exactly in practice. The errors of the 24 parameters are inevitable, resulting in the end positioning/orientation errors of industrial robots. Due to its popularity, the D–H model is not explained in detail in this study. For convenience, the kinematics model refers to the D–H model in the following of this paper.

The end load of an industrial robot may lead to measurable deformations and pose errors of the robot. The parts of the robot that deform include its links and joints. The robot joint deformation is mainly generated by the transmission parts inside the joints, such as reducers, gears, and drive belts. Joint stiffness describes the ability of each robot joint to resist deformation caused by external forces and moments [21].

Insufficient torsional rigidity of driving motors and reducers may also result in pose errors of an industrial robot. Therefore, assuming that each link is a rigid body, each joint is considered equivalent to a spring with an elasticity coefficient equal to a constant stiffness value K [29]. This paper uses the VJM method, based on the Jacobian matrix, for stiffness modeling. This method considers only joints as the primary sources of deformation in industrial robots [24].

For industrial robots, link gravities may also cause joint rotation errors and end pose accuracy/repeatability of industrial robots [31]. When analyzing joint flexibility errors caused by the link gravities of a six-DOF industrial robot, the deformation of the fifth and sixth joints is small and can usually be neglected because of their small gravities. In addition, the z-axis direction of the first link is parallel to the direction of weight, and its weight does not generate any moment in the z-axis direction for the first joint, considering most of industrial robots are mounted on the ground. Thus, the rotation error of the first joint due to weight is not considered in this study. In order to simplify the modeling of joint errors due to weight, the third and fourth links could be considered as a whole based on the usual structure of industrial robots, and the gravity center is located on the z axis of the fourth link. Therefore, only the weight of the second, third, and fourth links is included. The force analysis of the robot joints by link weight is shown in Fig. 3.

In the previous sections, the performance tests and their index calculations commonly used for industrial robots are introduced, as well as the kinematic model and the joint stiffness model of six-DOF serial industrial robots. In practical cases, the control system of an industrial robot controls its pose based on these models. However, the model parameters cannot be identified exactly, and these parameter errors are the main error sources of pose errors for industrial robots. In order to analyze the sensitivity of the positioning/orientation accuracy/repeatability tests in the ISO 9283:1998 Standard to the parameter errors of the kinematic model and the joint stiffness model, the simulation procedures of the sensitivity are developed in this section.

As addressed in Sec. 3.1, there are usually 24 kinematic model parameters for six-DOF serial industrial robots: a_i, α_i, d_i, θ_i (i = 1, 2, …, 6). The kinematics model of industrial robots is the basis of its control system, and affects robot motion performance during its use. In order to analyze whether the positioning/orientation accuracy/repeatability tests described in the ISO 9283:1998 Standard [2] are sensitive to those geometric error sources, the sensitivity simulation method is given as follows.

The simulation steps for the positioning/orientation accuracy/repeatability test sensitivity to the kinematic model parameters are shown in Fig. 5. It shows the error sensitivity calculation steps of a robot performance test index supposing that only a certain model parameter error occurs. First, the robot command pose is given according to the ISO 9283:1998 Standard, and the kinematics model parameters are initialized. Second, the model parameter error is set. According to the command pose, the actual output pose of the industrial robot is calculated based on the inverse kinematics and the model parameters with the error. Finally, this positioning/orientation accuracy/repeatability index in the performance test standard is calculated based on the commanded poses and the actual poses to obtain the error sensitivity relative to this model parameter. Based on the calculation method of positioning/orientation accuracy/repeatability in the standard, this paper obtains the error sensitivity matrix corresponding to the above 24 parameters after obtaining all of the sensitivities by simulation, as described in Sec. 4. After deriving the sensitivity matrix of the performance test to all kinematic model parameter errors, it can be found whether the current tests of ISO 9283:1998 are sensitive to the parameter errors of the kinematic model leading to positioning/orientation errors of industrial robots, as described in Sec. 5.1. If all the considered performance tests are insensitive to one or more error sources, the performance tests require improvement or supplementation.

Figure 7 shows the simulation procedure of positioning/orientation accuracy/repeatability test sensitivity to robot stiffness model parameter errors. First, the command pose is given, the needed joint angles are calculated based on the inverse kinematics, and the Jacobian matrix required for the stiffness model calculation is constructed. Second, the joint stiffness model considering the end external load and the self-weight of the links is established. In this step, the external load and the offset of the gravity center of the load are set according to the ISO 9283:1998 Standard. Then, the error caused by the joint stiffness model is compensated by modifying the joint angles. Finally, the sensitivity simulation of positioning/orientation accuracy/repeatability tests to the model parameter errors is performed. In this simulation, an error of one of the joint stiffness model parameters is set, and the performance test output is calculated based on the positioning/orientation accuracy/repeatability definition. This output is considered as one result of sensitivity. Thus, the sensitivity matrix of the performance test to all of the joint stiffness model parameter errors could be derived after setting the model parameter errors one after another. After deriving the sensitivity matrix of the performance test to all stiffness model parameter errors, it can be found whether the current tests of ISO 9283:1998 are sensitive to the parameter errors of the stiffness model leading to positioning/orientation errors of industrial robots, as described in Sec. 5.2.

When analyzing the sensitivity of industrial robot performance test methods to model parameter errors, it is essential to set reasonable values of parameter errors as error inputs. In this section, both the kinematic model parameters and the joint stiffness model parameters in terms of the six-DOF serial industrial robots are identified by several times based on the experimental data. Thus, several groups of the parameters of both the kinematic model and the joint stiffness model can be derived. And then, the standard deviation of each model parameter is obtained, which is used as the input of the error sensitivity simulation for each parameter.

In factories of industrial robots, the nominal or the calibrated D–H model parameters are input into the robot controllers. Actual D–H model parameters often differ from nominal/designed values because of different manufacturing and assembly qualities. The kinematic model parameters of industrial robots may also change due to their long-term use. Thus, the positioning/orientation accuracy/repeatability of industrial robots often deteriorates during their service lives. Table 2 shows the nominal D–H model of the ABB IRB 1410 robot.

In the experiment, a Leica AT403 laser tracker is used to measure 60 end points of the robot in its workspace, and the corresponding joint angles are read and recorded from the teach pendant of the robot. The experiment is shown in Fig. 8. Then, the iterative least-squares algorithm is used to identify the kinematic model parameters. Among the 60 groups of measured data, 80% of them are selected for identifying the D–H model parameters, and 20% of them are used as the validation data. The identified actual D–H model parameters are obtained and validated using the validation data. One of the groups of the absolute positioning errors after calibration is shown in Fig. 9. It can be found that the positioning accuracy based on the identified ABB robot kinematics model parameters has been significantly improved, all of which are less than 1 mm. This verifies the effectiveness of the identification method.

The D–H model parameters of the ABB robot are repeatedly identified 20 times in this study. In each identification, 80% of the data are randomly selected as the training data. The kinematic models are verified based on the condition that the positioning accuracy results (i.e., absolute positioning errors) after calibration are less than 1 mm. Thus, 20 groups of the identified kinematic model parameters are obtained, and each group contains the calibrated results of the 24 kinematic model parameters. The standard deviation of each parameter is calculated by the Bessel formula to estimate the overall error distribution of the model parameters, and the results are shown in Table 3. In the sensitivity simulation, the error of each model parameter d, a, θ, or α is individually set to its standard deviation value shown in Table 3, and the results of the performance test indexes in the standard could be calculated.

Additionally, the positioning/orientation repeatability error of industrial robots is caused randomly by friction, joint clearance, gear transmission, etc. The repeatability of the robot is the result of the comprehensive effect of these error sources. It is difficult to model and analyze random errors. Therefore, from the perspective of kinematic model parameters, the random errors are attributed to the joint angle errors to analyze their influence on the repeatability accuracy.

The joint angle errors are evaluated based on the experiment for ABB IRB 1410. In this experiment, one of the robot joints rotates alone, and the other joints remain motionless. The motion of each joint is repeated 20 times with different rotation angles, and the robot TCP positions are measured using the laser tracker. Thus, the mean and standard deviation of the repeatability error of each joint angle is calculated based on the measured data of the laser tracker, as shown in Table 4 for the ABB IRB 1410 industrial robot.

Then, in the simulation of the repeatability test, the Monte Carlo method is used to generate the joint angle errors randomly according to the result in Table 4. By taking 30 random samples based on the Monte Carlo simulation, the repeatability results of position and orientation are simulated and calculated based on the ISO 1998:9283 Standard.

Before this study, the model parameters of the ABB IRB 1410 robot joint stiffness are unknown and could not be obtained from the manufacturer. Usually, two methods of identifying the joint stiffness model of industrial robots are often used: the calculation method according to an empirical formula and the identification method based on a loading experiment. In this study, the joint stiffness model parameters of the ABB IRB 1410 robot are identified experimentally by using the Leica AT403 laser tracker. Twenty groups of the robot poses are selected, and their end positions are measured. On the other hand, two types of loads, 2 kg and 4 kg, are fixed to the robot end link, respectively. In this experiment, the laser tracker is used to measure the end positions of the robot under no-load, the 2 kg load, and the 4 kg load, respectively. Therefore, 40 groups of the robot TCP deformation under different loads are measured. The experiment is shown in Fig. 10. According to the force-stiffness-deformation relationship derived above, the joint stiffness model parameters of the robot are then identified by the least-squares optimization algorithm based on the deformation data and the joint angles.

It should be noted that the kinematic model parameters of the robot are needed during the identification of its joint stiffness parameters. Therefore, its kinematic model parameters should be calibrated first. Additionally, in this experiment, the robot is loaded only in the gravity direction, and the axis direction of the robot first joint is parallel to the gravity direction. Therefore, the first joint is not involved in the stiffness identification process. Furthermore, among 40 groups of the end position data, 30 of them are selected randomly in each identification, and remaining 10 groups are used for verification. Totally, 20 times of identification are performed, and 20 sets of the joint stiffness model parameters can be derived. Finally, the mean and the standard deviation of each joint stiffness parameter are calculated, as shown in Table 5.

In order to verify the identified joint stiffness model further, 10 groups of different poses are selected randomly again. The laser tracker is used to measure the end deformation of the robot under loads. On the other hand, the deformation value is also calculated using the identified joint stiffness model with the mean parameters shown in Table 5. The comparison results are shown in Fig. 11. It can be found that there is a small gap between the calculated positions based on the calibrated stiffness model and the measured end positions. The reason is that the stiffness model is based on the lumped parameter method concentrating on the deformation of each joint to a certain point without considering the influence of other factors. Based on the results shown in Fig. 11, the identified stiffness model can be considered reliable in this study.

Finally, the sensitivity analysis of the performance tests on the joint stiffness model is performed by setting the parameter errors of the joint stiffness model as the standard deviations shown in Table 4.

By the sensitivity simulation, it can be analyzed whether the robot performance tests specified in the ISO 9283:1998 Standard can be sensitive to the error sources from the robot kinematics model and the joint stiffness model. In this study, the ABB IRB 1410 industrial robot is taken as an example, and a matlab robot toolbox is used for the simulation.

In the ISO 9283:1998 Standard, all tests of industrial robots should be located in a plane of the cube, introduced in Sec. 2. The simulation is performed based on these rules of the standard. Considering the angle ranges of the robot joints and the reachable space of the ABB robot end effector, as shown in Fig. 12, the acceptable cube and the test plane are selected in the workspace based on the standard. The five measurement points P₁, P₂, …, P₅ are located on the diagonals of the test plane, which are obtained by the selected plane with an axial measurement point offset and a radial measurement point offset. The positions of the five test points under the base coordinate system are measured by the measurement instrument, and the rotation angles in terms of the fixed axes of the base coordinate system are used as the orientation angles. Table 6 shows the positions and orientations of the selected five test points. The tests of the pose accuracy and repeatability specified in the standard should be with 100% of the nominal load and 100% or 50% of the nominal speed. The TCP of the robot should pass through the five points in a same approaching direction, and the movement should be repeated 30 times. Based on the ABB robot manual, it can be found that its nominal load is 5 kg. According to “Table A.2—Standard test load categories” in the ISO 9283:1998 Standard, the axial center of gravity (CG) offset is set to 60 mm, the radial CG offset is set to 30 mm, and the axial measurement point offset is set to 120 mm. Finally, the error sensitivity is obtained by substituting the command pose and the actual pose into the calculation model of the performance indices.

Based on the simulation procedure given in Fig. 5, the sensitivity simulation of the positioning/orientation accuracy/repeatability test indexes to model parameter errors is performed.

In Sec. 4.3.1, the actual standard deviation of each parameter of the kinematic model is calculated by identifying multiple groups of the model parameters for the ABB IRB 1410 robot. Therefore, the sensitivity simulation of the positioning/orientation accuracy tests to the standard deviation errors (shown in Table 3) of the kinematic model parameters is performed. While in the sensitivity simulation of the positioning/orientation repeatability tests, the errors of the joint angles with the means and the standard deviations (shown in Table 4) are generated randomly based on the Monte Carlo method. The simulation results are shown in Tables 7–10.

Nowadays, maximum permissible errors of laser trackers given by their manufacturers can be less than 20 μm when measured points are close to laser trackers. As a result, the threshold value for the sensitivity analysis is set as 0.02 mm. It is considered that the sensitivities are too small to be effective when they are less than this threshold. In such a case, the corresponding test is considered non-sensitive to the corresponding model parameter errors. In Table 7, the positioning accuracy test sensitivities to the errors of the joint offset d₁ and the link length a₁ of joint 1 can be identified non-sensitive, and the positioning accuracy test sensitivities at point P4 and point P5 to the error of the twist angle α₂ of joint 2 can also be regarded as non-sensitive. Usually, the measurement instruments used for the positioning accuracy test of industrial robots generally require high accuracy and are expensive, such as laser trackers. Therefore, most industrial robot users can only perform the positioning repeatability test. In the repeatability test, the random errors caused by joint clearance, gearing, etc. are represented by the joint angle errors. However, the repeatability is not sensitive to the link length, the twist angle, and the joint offset completely. Therefore, the error sensitivity to them can be directly identified as zero without simulation, which is represented by “/” in Tables 7 and 8. The error sensitivity results above show that the positioning repeatability test has great limitations for industrial robot performance evaluation.

Tables 9 and 10 show that the sensitivities of the orientation accuracy test to the errors of each joint link length a_i and each joint offset d_i are zero. This indicates that the robot orientation accuracy test results do not reflect the errors of the link length a_i and the joint offset d_i. In the orientation repeatability test, the errors of the link length a_i, the twist angle α_i, and the joint offset d_i also do not affect the test results based on the kinematic control method of industrial robots. Therefore, their error sensitivities can also be directly identified as zero without simulation, as indicated by “/” in Tables 9 and 10. Due to the need of high-accuracy instruments, the industrial robot orientation accuracy and repeatability tests are less used in practice, but they still cannot be ignored. Based on the results above, it can be seen that the six-DOF serial industrial robot orientation accuracy and repeatability tests cannot reflect the errors of the robot kinematic model parameters including d_i and a_i, i = 1, 2, …, 6. Using these two test indexes alone to evaluate industrial robots has obvious limitations. For the six-DOF serial industrial robots, there is little point in performing orientation accuracy and repeatability tests when positioning accuracy and positioning repeatability tests can be performed.

Besides the ABB IRB 1410 robot, we also investigate the performance test sensitivity to the kinematic model parameter errors for the UR5 robot. The UR5 kinematic parameters are also calibrated several times, and their standard deviations are also derived by the experiment shown in Fig. 13. Then, based on the simulation procedure shown in Fig. 5, the standard deviations of the parameter errors are input as the error setting for the sensitivity simulation of the performance tests to the kinematic parameters. Finally, it is found that the obtained simulation results are similar with those of the ABB IRB 1410 robot shown from Tables 7 to 10. This finding shows that the conclusions of the sensitivity analysis of the pose accuracy tests for six-DOF serial industrial robots above are somewhat generalizable.

Currently, many users of industrial robots only perform the positioning repeatability test due to the limitation of measurement instruments. The analysis above shows that the positioning accuracy test of industrial robots is very necessary. On the other hand, the tests of orientation accuracy and orientation repeatability of industrial robots have limitations for evaluating their motion performance. The orientation accuracy and repeatability tests often require a long time and high-accuracy instruments, which may be neglected if the positioning accuracy and repeatability tests can be performed, considering that most of industrial robot positioning/orientation errors are incurred by geometric errors.

Figure 7 shows the sensitivity simulation procedure of the pose performance tests when considering the stiffness model parameter errors of industrial robots. Based on this procedure, the error sensitivity simulation of the positioning and orientation accuracy tests is done for the ABB IRB 1410 robot in this study. The sensitivity analysis results are shown from Tables 11 and 12.

Tables 11 and 12 show the sensitivity matrix results of the positioning/orientation accuracy when the stiffness error is set to the standard deviation of each identified parameter of the stiffness model. From the sensitivity results in these tables, it can be seen that the sensitivity of the positioning/orientation accuracy tests to the robot stiffness model parameter errors are different. The sensitivities of stiffness parameter errors of the second and third joints are larger compared with those of other joints, indicating that positioning/orientation accuracy tests can better reflect the stiffness parameter errors of the second and third joints. It also indicates that manufacturers of six-DOF serial industrial robots should pay special attention to the stiffness of the second and third joints during the designing and manufacturing process.

As shown from Tables 11 and 12, both the positioning accuracy and the orientation accuracy are less sensitive to stiffness errors of the fourth, fifth, and sixth joints. This indicates that the test results of the positioning accuracy and the orientation accuracy cannot reflect the joint stiffness model errors of the fourth, fifth, and sixth joints in the control systems of industrial robots. The small sensitivity of positioning accuracy may be decided by the robot mechanical structure: the fourth, fifth, and sixth joints near the end and its link length a_i and joint offset d_i are relatively small; the stiffness errors of the fourth, fifth, and sixth joints mainly affect the end orientation.

The reason for this conclusion is that the link lengths a_i and the joint offsets d_i in terms of joints 4, 5, and 6 are small or zeros. Considering the mechanical structure of the robot, the first, second, and third joints of the traditional six-DOF serial industrial robot make the end of the robot reach the approximate target position. The fourth, fifth, and sixth joints belong to the wrist structure, and their main function is to make the robot end effector achieve the desired orientation. However, the orientation accuracy test specified in the ISO 9283:1998 Standard does not also reflect the joint stiffness model parameter errors well. The measurement of the end orientation of industrial robots is currently cumbersome. For six-DOF serial industrial robots, it is of little significance to perform orientation accuracy tests.

The first joint stiffness is not involved in this sensitivity simulation. In the simulation of this study, the ABB IRB 1410 robot is fixed on the horizontal floor. Thus, the rotation axis of the first joint is parallel to the gravity direction in the joint stiffness model. In the static stiffness model, the torque around the rotation axis due to the load gravity to the fitst joint is zero. Therefore, the stiffness of the first joint could be ignored in this study, as done in other related research [36]. The ISO 9283:1998 Standard does not address in detail the effects between the robot fixing position and the forces on each robot joint under end loads, while only suggesting that the robot should be mounted according to the manufacturer's recommendations. The standard does not specify the direction of the load force applied to the end flange of the robot. Most manufacturers or researchers perform robot performance tests by the end load with the gravity direction. The installation position of the robot is on the horizontal ground, which leads to the rotation axis of the first joint parallel to the load gravity direction during the performance test. As a result, the error sensitivity of the positioning/orientation accuracy/repeatability tests to the fitst joint stiffness is zero.

Tables 11 and 12 also show that the selection of the test points from P₁ to P₅ has an influence on the test results of the pose accuracy when the joint stiffness error is the same. In this simulation result, the error sensitivities of the tests at the points P₂ and P₃ are relatively larger, which indicates that the selected points P₂ and P₃ in the spatial cube are good test points and can better reflect the joint stiffness model parameter errors. It could be concluded that the selection of the test cube in the robot performance test has a clear impact on the test results. The ISO 9283:1998 Standard only specifies that the test cube should be located in the part of the workspace where the most applications are expected, but without more detail. This leads to the possibility that robot manufacturers may select test points with better test results, making the performance test results nonuniversal.

According to the sensitivity analysis of the position/orientation accuracy tests to joint stiffness parameter errors, it can be seen that these tests also have some limitations for evaluating the static stiffness model parameter errors in controllers of industrial robots. Although the position/orientation accuracy tests have good sensitivities to the stiffness parameter errors of the second and third joints, but poor sensitivity to the stiffness parameter errors of the fourth, fifth, and sixth joints, they cannot reflect the stiffness error of the first joint.

The sensitivity analysis of the positioning/orientation accuracy tests on joint stiffness model parameter errors above is based on the ABB IRB 1410 robot with a normal load of 5 kg, which is a small-load type industrial robot. In Ref. [30], the authors investigated the joint stiffness of the KUKA KR 240 industrial robot, which is a large-load robot with a normal load of 240 kg. Its maximum reach is close to 3 m. In this study, the sensitivity simulation to the stiffness parameter errors of the KUKA KR 240 industrial robot is also performed based on the model parameters identified by the authors in Ref. [30]. The input errors in this simulation are set to 10% of each joint stiffness parameter given in Ref. [30]. In the simulation result, the error sensitivities of the KUKA industrial robot under the normal load are larger compared with those of the ABB IRB 1410 robot. The positioning and orientation accuracy tests can reflect the joint stiffness parameter errors of the second, third, and fifth joints. It is also found that the positions of the test points have obvious effects on the sensitivities when a certain joint stiffness error of a joint is input in the simulation. Since the simulated KUKA robot is still fixed on a horizontal ground and the rotation axis of the first joint is also parallel to the gravity direction, the first joint stiffness is also not involved in the test. In such a case, the positioning/orientation accuracy test in the ISO 9283:1998 Standard is also ineffective for exploring all of the joint stiffness errors of the KUKA robot.

At present, there are joint stiffness compensation algorithms and programs in many control systems of six-DOF serial industrial robots, which is beneficial for higher-precision pose control. The sensitivity simulation results of the positioning/orientation accuracy tests to joint stiffness parameter error show that: (1) The test methods of the positioning/orientation accuracy cannot completely reflect the robot joint stiffness model parameter errors based on the ISO 9283:1998 Standard, and they have limitations for evaluating joint stiffness compensation performance of industrial robots. (2) At different test points, the sensitivity results to the same joint stiffness parameter error are obviously different, which indicate that the selection of the test cube and the test points is important. It is suggested that the ISO 9283:1998 Standard could describe the locations of test points more clearly.

In this subsection, further simulation analysis of the distribution of positioning accuracy test sensitivities to parameter errors of the kinematic model and the joint stiffness model in the robot workspace is done. In this simulation, a random sampling method is used to obtain 2000 groups of randomly generated joint angles of the ABB industrial robot. The sensitivities of the positioning accuracy test to the model parameter errors for the 2000 test points are simulated and calculated based on the simulation steps in Figs. 5 and 7. The result is called the spatial distribution of the error sensitivities of the positioning accuracy test, which could be plotted in the robot workspace.

Figures 14 and 15 show the spatial distribution of the sensitivities of the positioning accuracy test to the joint angles θ₁ and θ₂ in the kinematic model parameter errors, respectively. It can be seen that the sensitivities to the first joint angle θ₁ are generally larger when they are farther from the Z axis of the robot base coordinate system based on Fig. 14. In Fig. 15, it is found that the farther the test position is from the center point of the workspace, the greater the error sensitivity to the second joint angle θ₂ is. Additionally, the spatial distribution characteristics of the sensitivities of the positioning accuracy test to other kinematic parameter errors is also simulated, as summarized in Table 13. As shown in this table, the sensitivities of the positioning accuracy test to the joint offsets and the link lengths are equal everywhere in the robot workspace. Only the sensitivities to the joint angles and the twist angles of the first and second joints have obvious distribution characteristics.

On the other hand, the spatial distribution of the sensitivities of the positioning accuracy test to the joint stiffness model parameter errors is also analyzed based on the simulation. It is found that there are obvious spatial distribution patterns in terms of the sensitivities of the positioning accuracy test to the second and third joints stiffness parameter errors, shown in Figs. 16 and 17, respectively. The sensitivities to the second joint stiffness error are generally greater further away from the center of the robot workspace. While the sensitivities to the third joint stiffness error are generally larger in the middle of the workspace when the test points are further away from the Z axis, and they are smaller at the upper and lower portions of the robot workspace. The distribution characteristics of sensitivities of the positioning accuracy test to the stiffness model parameter errors are obtained and shown in Table 14.

Thus, in order to better evaluate the positioning accuracy of industrial robots, besides the five points P₁–P₅ recommended by ISO 9283:1998, additional test points may be suggested to be far away from the Z axis of the robot base coordinate system and relatively high based on the simulation above.

The positioning/orientation accuracy/repeatability calculation and test methods specified in the ISO 9283:1998 Standard for industrial robots are described in Sec. 1. In the ISO 9283:1998 Standard, the test cube is determined in the workspace where the robot is expected to be used the most, and the test points are restricted to the diagonals of the test plane within the cube, which are with five different poses to characterize the robot positioning/orientation accuracy/repeatability. It reflects that the robot approaches the command pose from the same direction, and the deviation between the command pose and the average value of the actual measured pose. Given that the robot positioning errors may be related to the positions, the test methods specified in ISO 9283:1998 cannot evaluate the robot performance very well.

In the previous section, the spatial distribution of the error sensitivities of the positioning accuracy test to the parameters of the kinematic model and stiffness model is analyzed. It is found that the position accuracy sensitivity is larger when the position is far from the Z axis of the robot base coordinate system and relatively high. In this section, an improved method is proposed as the positioning accuracy test in order to find which positions in the workspace of actual industrial robots can reflect error sources better. A 180 deg semi-circular arc is determined as high and far as possible, and it is above the test cube given in the ISO 9283:1998 Standard. The seven points with the intervals of 30 deg on the arc are taken as the additional designed test points. As found in Sec. 4.3, the positions with larger positioning errors by parameter errors of the kinematic and stiffness models are distributed in the higher and farther portions in the robot workspace. This means that these model parameter errors are more observable in such test positions. The additional designed test points in this study are located higher and farther above the test cube in the ISO 9283:1998 Standard, being close to the workspace boundary area. As shown in Fig. 18, the given test points are evenly distributed in the 180 deg forward area of the first joint, which may capture the pose accuracy of the industrial robot as completely as possible.

The ABB IRB 1410 robot is used as the test object, as shown in Fig. 18. According to the ISO 9283:1998 Standard, the robot's positioning accuracy is tested first. The test points P₁–P₅ based on this standard are identified, while the actual positions are measured using the Leica AT403 laser tracker, and the measurement is repeated 30 times. After finishing the test experiments specified in the standard, let the ABB robot to reach the seven additional test points M₁, M₂, …, and M₇ designed in this study, as shown in Fig. 18. The seven test points should be close to the workspace boundary area, being higher and farther relative to the test cube. The coordinate measurement of the seven test points M₁, M₂, …, and M₇ is also repeated 30 times. According to the ISO 9283:1998 Standard, we also added a 4.23 kg load to the end of the robot during the whole experiment, and the experiment is shown in Fig. 19.

According to positioning accuracy calculation formula given in the ISO 9283:1998 Standard, the results of the position accuracy are shown in Tables 15 and 16. By comparing Table 15 with Table 16, it can be found that positioning accuracy result based on the test in the ISO Standard is smaller in general than the position accuracy derived based on the additional designed test points in this study. The larger positioning accuracy based on M₁, M₂, …, and M₇ indicates that they can reflect the error sources better, i.e., the positioning accuracy test based on them is more sensitive to the error sources. The averaged positioning accuracy result of the proposed test points in Table 16 is 2.237 mm, which is 37.4% larger compared with the averaged position accuracy result of 1.628 mm based on the ISO Standard in Table 15. It is shown that the proposed additional test points provide better observability of error sources of the actual industrial robot than those specified in the ISO standards.

The similar tests are also conducted on the UR5 robot, as shown in Fig. 20. It is a popular lightweight industrial robot. The laser track is also used for measuring the positions of the test points. The averaged positioning accuracy of the proposed seven test points is 3.1159 mm, which is also a significant improvement compared with the averaged positioning accuracy of 2.5677 mm based on the test points in the ISO 9283:1998 Standard. The result also indicates that the suggested test method in this paper has better observability for error sources.

The proposed performance test method above may be a useful complement to the positioning test method in the ISO 9283:1998 Standard for six-DOF serial industrial robots. Especially for those productions of industrial robots without determining the working portion in the workspace with the greatest anticipated use, it may be beneficial to evaluating their positioning accuracy based on the proposed seven test points above.

In this study, in order to find the effectiveness of the performance tests of industrial robots in the ISO 9283:1998 Standard, the sensitivities of the positioning/orientation accuracy/repeatability tests to parameter errors of the kinematic model and the joint stiffness model are simulated and analyzed for six-DOF serial industrial robots. The kinematic model and the joint stiffness model are usually embedded for implementing motion control in industrial robot control systems. In this paper, the kinematic model and the joint stiffness model of the ABB IRB 1410 robot and the UR5 manipulator are identified first. The parameter standard deviations of the two models are obtained, which are used as the input errors in the sensitivity simulation. Second, the simulation procedures of the sensitivities are described. Based on the tests given in the ISO 9283:1998 Standard, the sensitivities of the position/orientation accuracy/repeatability tests are simulated, and the sensitivity matrices are also presented. The sensitivities of the performance tests to the model parameters are described in detail. Finally, an improved positioning accuracy test is proposed by analyzing the spatial distribution characteristics of the sensitivity of the positioning accuracy test. And the experiments are conducted to measure the positioning accuracy of the test points based on the ISO 9283:1998 Standard and the proposed test. The main conclusions are as follows:

1. For six-DOF serial industrial robots, the positioning accuracy test is necessary with the consideration of evaluating parameter errors of the kinematic model and the joint stiffness model in control systems, which are the key to determining their motion performance. The positioning repeatability test of the ISO 9283:1998 Standard cannot reflect the errors of link lengths, twist angles, and joint offsets of the kinematic model in control systems of industrial robots. Additionally, based on the orientation accuracy test in the standard, not all of the joint stiffness of industrial robots can be evaluated. It may be concluded that the orientation accuracy and repeatability tests are not needed if the positioning accuracy and repeatability tests can be done for six-DOF serial industrial robots.

2. In the ISO 9283:1998 Standard, a test plane and five test points are suggested based on the planned working portion in the robot workspace with the greatest anticipated use. However, the locations of the five test points are not strictly defined in this standard. Based on the simulation results of the sensitivities for six-DOF serial industrial robots in this study, the test points should be located farther away from the Z axis of the base coordinate system and they are also recommended to be relatively high in the positioning accuracy test. Thus, the modeling errors of the kinematic and the stiffness characteristics of six-DOF serial industrial robots can be tested more easily.

In this paper, the sensitivity simulation method is proposed in order to analyze the validity of the performance tests in the ISO 9283:1998 Standard. A test method for the positioning accuracy is proposed and verified experimentally. The proposed test method could be a useful complement to the positioning test method in the ISO 9283:1998 Standard for six-DOF serial industrial robots. It is worth noting that this study focuses solely on six-DOF serial industrial robots, and thus the above conclusions might be specific to this particular type of robots. However, besides the focused positioning/orientation accuracy/repeatability tests of six-DOF serial industrial robots in this study, all the tests given in the ISO 9283:1998 Standard could also be further investigated for serial/parallel/Selective Compliance Articulated Robot Arm (SCARA) industrial robots based on the presented analysis scheme of this paper in the future. Furthermore, a sensitivity analysis of the performance tests in the workspace near six-DOF serial industrial robot singularities is also necessary. As a result, the ISO 9283:1998 Standard could be improved in the future. The research of this paper is helpful to improve the performance evaluation methods of industrial robots, and can also help robot manufacturers to analyze and improve the motion performance of their products.

• This study is partially supported by the National Natural Science Foundation of China under Grant 62373339.

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.