Uncertainty Analysis and Characterization of the SOFAST Mirror Facet Characterization System

The fringe reflection technique employed by SOFAST is a dynamic target method that determines surface normals at many points on a facet simultaneously and is described in the companion paper [1]. Others have reported using fringe reflection (also called deflectometry) for characterization of solar mirror facets and assemblies [3,4], providing highly detailed analysis of the mirror surface normal.

Correctly calculating the surface normals of the mirror facet requires accurate measurements of the target (TV monitor) dimensions, camera position relative to the target (translation and rotation), distance from the target to the facet, and the intrinsic camera parameters since a real camera and lens are not accurately represented by a perfect pinhole model (e.g., the camera skews incoming light which induces error in SOFAST calculations). The target dimensions and relative positions of the target/camera are quantified with simple tape measurements, and a laser distance finder is used to measure the distance from the facet to the target. Camera imperfection is approximated by quantifying the camera's focal length in two axes and estimating the lens distortion with four terms—two radial (barrel) and two tangential. These camera parameters are quantified via a calibration process [5]. The above-mentioned setup and camera calibration parameters are the inputs in the sensitivity/uncertainty study presented here. Another source of potential error is the uncertainty of accurately mapping pixels on the camera array to locations on the target as mentioned above. The use of a TV monitor is possible for dish facets having a relatively short focal length and allowing placement of the monitor at the 2f location. Since the TV monitor has a fixed and accurate pixel pitch, the expected uncertainties can be significantly smaller than systems relying on a projector and screen for the active target. The larger target produced by a projector is required for flatter facets for trough and heliostat applications.

The data collected by SOFAST are converted to slope data and then fitted to Eqs. (3) and (4) via least squares. Facet focal lengths are represented by 1/(4A) and 1/(4B). For small facet tilts, the C and D terms can be interpreted as the tilt of the facet. The tilt terms and piston (constant) term are numerically driven to values related to the measured focal lengths and twist terms such that a specific point on the measured facet matches the design facet in position and rotation [1]. Therefore, they are dependent terms and not independent outputs. The E term, for a full parabola of revolution on a unit circle, can be interpreted as the astigmatism at 45 deg relative to the measurement axis [6]. However, for a partial parabola—such as the ADDS facet—this term can also be interpreted as the end-to-end facet twist [1]. Global slope error, more correctly termed the residual, is the root-mean-square (RMS) across all facet points of the difference between the measured data and the model parabola. It is important to note that this model may be an ideal design parabola or, in the case presented here, a fitted facet shape. As such, it is critical that slope error residuals not be divorced from the model to which they are residual. It is also important to emphasize that, if the residual is to be used as a normally distributed error in a ray-trace or other model, this residual must truly be distributed normally and not contain systematic errors.

Local slope error, also reported as an output in this study, is the slope deviation measured at one point relative to the modeled surface. Local slope error is reported for a handful of points from the mirror facet in this study, since local deviations are lost when considering global slope error.

SOFAST and other deflectometry tools mentioned here provide extensive facet characterization data. It is important to determine how uncertainties in system inputs affect system outputs for each tool in order to understand the confidence intervals for facet measurements reported by each tool. An uncertainty analysis of the Video Scanning Hartmann Optical Tester (VSHOT) [7,8] system indicated the uncertainty in the fitted facet focal lengths and the local and global (RMS) residual (slope error) [9]. The uncertainty was measured relative to a high quality telescope mirror. Focal length uncertainty was determined to be ±0.8%, and the RMS residual slope error was accurate within ±0.1 mrad, though at a limited number of points. Marz et al. [10] performed an empirical uncertainty analysis of the CSP Services Deflectometry system for the measurement of trough facets. The stated uncertainty was 0.5 mrad local uncertainty in the slope measurement, and <0.2 mrad RMS, determined by imaging a flat surface of water. Marz does not report an uncertainty in focal length fit, since the CSP Services system does not report a fitted focal length. The CSP Services system uses a projection screen target to capture data from a long-focal-length facet, such as heliostat and trough facets.

To quantify the sensitivity and uncertainty of the point-focus version of SOFAST, an ADDS facet, whose design focal length (f) is 5.33 m, was measured at three distances: 2f, 2f + 2 m, and 2f + 4 m. These data provided a baseline for the carefully measured SOFAST setup. Facet measurement distances farther than 2f were considered because these distances result in larger reflected target images and, therefore, will likely have marginally different sensitivities than measurements made in the 2f region. Facet measurements were not made closer than 2f since doing so would have required a shorter focal length lens for the camera (due to field of view constraints). Switching camera lenses during the analysis would have introduced a host of additional variables and was avoided.

Fringe data were taken at each of these locations and then reprocessed through SOFAST as each input parameter was varied across a reasonable error band in fifty-one evenly spaced increments. The variation in SOFAST output parameters was tracked as each input parameter was varied and sensitivities were calculated in each case. Table 1 outlines the input parameters, output parameters, and their nominal values at the 2f measurement location.

Tables akin to Table 1 for the two other facet measurement distances of 2f + 2m and 2f + 4m are not included here as the only value that changes is the distance from target to facet. Thus, calculating sensitivities for every input at each distance required 2295 SOFAST runs (15 Inputs*3 Distances*51 Increments for each input). Though 16 inputs are listed, camera focal lengths were varied together to mimic a camera scaling error. These 2295 SOFAST runs resulted in 585 distinct sensitivity values (15 Inputs*13 Outputs*3 Distances). This is notable as it will drive how results are presented in the Results sections.

Setting reasonable uncertainty bounds for each input parameter was achieved by analyzing the means by which the input measurements were made. Intrinsic camera calibration parameters (focal lengths and lens distortion coefficients) were calculated by imaging a checkerboard of known dimensions and processing these images with a piece of Sandia-developed software that is based on the CalTech calibration toolbox [5]. Estimates for camera calibration parameters provided by the toolbox also include uncertainty (standard deviation) estimates. These camera uncertainty estimates were used in the SOFAST analysis presented here.

Beyond camera calibration parameters, other inputs included physical measurements of the target dimensions, relative camera/target translations and rotations, and the distance of the facet relative to the target. Target dimensions and target/camera translation measurements were made with a tape measure and were assumed to be accurate within ±3 mm. The distance from the facet to the target was measured with a laser distance finder whose uncertainty is published to be ±1.5 mm [11]. Despite this, ±3 mm was used in order to remain conservative.

Target/camera relative rotations were actively driven to zero during setup by squaring the camera to the target. Pitch and yaw were zeroed using a laser square that projected a point onto a wall in front of the camera that represented the intersection of two planes orthogonal to the target. A preview of the camera with a crosshairs superimposed on the field of view was then used to align the camera to the laser projected point. The test setup was squared at a working distance of roughly 15 m. Assuming the alignment was within 0.05 m in pitch and yaw at this distance gives an uncertainty of ±3 mrad. Roll was set by leveling the target then finding a horizontally level fixture (confirmed by measurement) in the field of view of the camera. The camera was then manually rolled until the horizontal crosshair was aligned to the level object in the field of view. Assuming one is able to sight a 1.5 m fixture and horizontal crosshair to within 0.025 m accuracy at a distance of 3.7 m leaves an uncertainty of ±7 mrad in roll.

Finally, pixel mapping—the accurate mapping of camera pixels to locations on the target—was assumed to be accurate within ±2 pixels based on controlled tests run on the SOFAST system. Table 2 summarizes the uncertainty bands for each input.

Sensitivities were calculated for every output variable with respect to every input variable at each measurement location. As mentioned above, this was achieved by varying each input in 51 evenly spaced increments across the uncertainty bounds shown in Table 2. SOFAST characterization outputs were logged in each case and sensitivities were calculated by finding the slope of the output parameter with respect to the variation in input parameter. Figure 3 illustrates the sensitivity of facet focal length in the x-direction to all 14 physical setup and camera calibration input parameters at measurement location 2f.

As shown in Fig. 3, facet focal length, x-direction is most affected by variations in horizontal target size and the target to facet distance. This is intuitive as the horizontal size of the target scales in the same direction as the x-direction facet focal length. The target to facet distance having a noticeable effect on x-direction facet focal length is also intuitive as this distance directly corresponds to a change in focal length. Regardless, x-direction facet focal length never varied more than 2 mm for any input parameter variation. Focal length variations resulting from input parameters for both focal length directions at all three measurement locations looked very similar to the results shown in Fig. 3. As such, plots for facet focal length variations in the y-direction or at other measurement locations are not included here.

Figure 4 shows the effect on facet twist in the x-direction as a result of input parameter variation at measurement distance 2f. The only input parameter with noticeable impact on the x-direction facet twist is the relative rotation of the target and camera in the z-direction, or camera optical axis. This parameter is the roll of the target relative to the camera (see Fig. 1). Adding roll between the camera and target manifests itself as additional twist because the reflected fringe patterns are not orthogonal to the camera pixel plane as expected by SOFAST. Twist results in both directions at all three measurement locations were very similar to the results shown in Fig. 4. As such, no additional plots are included here.

Figure 5 shows the effect on global slope error caused by input parameter variations. As mentioned above, global slope error is the RMS of the difference in measured data to the model parabola for all points measured on the facet. All input parameters had some noticeable effect on the result with horizontal target size and camera lens barrel distortion corrections showing the largest impact. Across the board, global slope error was relatively insensitive to input parameter variation. Global slope error sensitivity results at the other two measurement locations were very similar to the results shown in Fig. 5 and, as such, plots are not included here.

Figures 3–5 display the general nature of facet focal length, facet twist, and facet global slope error sensitivities to input parameter variation. Also of interest is the local slope error at a point (instead of a lumped global representation) as many in the concentrating solar power (CSP) community are concerned about calculating intercept factors and other similar ray tracing activities. To assess the impact of input parameter variation on local slope error, eight points on the facet were selected (one per sub mirror) and local slope errors in each direction (x and y) were logged for each input variation. Figure 6 illustrates the points selected for this analysis.

Figure 7 shows the sensitivity of the local slope error in the x-direction for facet point one measured at the 2f location. All inputs have some noticeable effect but, in general, the local slope error in the x-direction at facet point one is largely insensitive to input parameter variation.

Figure 8 shows the same data but in the y-direction. Local slope error in the y-direction was primarily affected by camera/target relative rotation in the z-direction (roll). In both cases (x and y directions), the effects of input parameter variation on local slope error were fractions of a milliradian.

The nature and magnitude of the local slope error sensitivities for both x and y directions held for all eight facet points at all three facet measurement locations. Figure 9 shows the sensitivity of the local slope error magnitude to input variations at facet point one for measurements made at 2f. These values were calculated by a root-sum square of the two local slope error components.

As seen in Fig. 9, local slope error in the composite sense was most affected by variations in horizontal target size. Effects of camera/target roll were smaller given the relatively smaller magnitude of the slope error value in the y-direction, which is likely caused by the significantly smaller target extent in the y-direction.

The analysis to this point has assumed that the mapping from each camera pixel to a location on the target is perfectly accurate. As there will be uncertainty in this mapping, uncertainty was added to the assumed-perfect slope data and sensitivities were evaluated.

At a facet measurement distance of 2f, the local slope error (single point) was found to have a sensitivity of 0.037 mrad for each pixel of mapping error imposed in both the horizontal and vertical directions. Discrete local errors imposed on just eight of the pixel return locations had no impact on the determined facet focal lengths, twists, or global (RMS) slope error residual. Conversely, when a normal distribution of error was applied to all data points, the sensitivity of local slope errors was about 0.01 mrad/pixel in both directions over a range of 0–10 pixels. However, the sensitivity appears to be parabolic, and is only 0.002 mrad/pixel over a range of 0–2 pixels of imposed error. The sensitivity to target pixel location errors was only performed at the nominal 2f location and would be correspondingly reduced by increasing the distance to the facet. Thus, for this study, pixel mapping error sensitivity was taken to be zero for facet focal lengths, twists, and global slope error because the impact was so small. For local slope errors, pixel mapping error sensitivity is taken to be 0.002 mrad/pixel and an uncertainty of ±2 pixels is assumed. Assuming ±2 pixels for local slope error pixel mapping uncertainty is supported by the fact that finer and finer fringe patterns, used to refine the mapping of camera pixels to target locations, resulted in less than one pixel location change for representative target points.

The uncertainty analysis presented here assumes all output parameter sensitivities to input parameter variations are linear, all input parameters are independent, and all input parameters are uniformly distributed across conservative bounds derived from an analysis of the SOFAST setup process. Results presented above show that linearly approximating input/output sensitivities is accurate. Assuming input parameter independence is a reasonable first step—future work will assess output uncertainty due to correlated input variation effects. Finally, assuming uniform distributions for each input parameter is inherently conservative and is the only defendable approach since actual distributions have not been derived for each input.

Equation (9) describes the variance of the output of interest, y_i, as a function of the input sensitivities and variances. Since all inputs, x_i, have been assigned uniform distributions, Eq. (10) describes the variance of these random variables. Finally, the overall standard deviation of the output parameter y_i is simply the square root of Eq. (9).

Table 3 illustrates the effect of each input on the variance of each output at measurement location 2f. The product of each input's sensitivity squared and variance (per Eq. (9)) is normalized relative to all other inputs for the output parameter of interest. The table quantifies the results displayed in the sensitivity plots above.

Table 3 illustrates the impact of each input on the variance of each output. Confirming the plots presented in the sensitivity section above, facet focal lengths were most sensitive to target dimensions and target/facet distance. Twist measurements were insensitive to all inputs except target/camera rotation in the z-direction (roll). RMS slope error was most sensitive to camera barrel distortion parameters and the horizontal target dimension. Local slope error in both directions at facet point one was most sensitive to pixel mapping error as target pixel location mapping errors directly affect facet normal vectors. Results for the other seven facet points were similar to those shown in Table 3 for facet point one. Results as those shown in Table 3 for facet measurements made at the other locations followed the same trends and are not included here.

Table 4 shows the composite uncertainties for each output at each measurement location. These composite uncertainties were calculated per Eq. (7) in the methodology section. Max local slope errors represent the maximum local slope error for all eight points shown in Fig. 6.

Increasing measurement uncertainty with increasing facet to target measurement distance is seen for facet focal length in the y-direction and twist in the x-direction. This is likely caused by the larger reflected image size that occurs as the facet measurement distance departs from 2f.

Using worst-case data for the uncertainties shown in Table 4, error bands were calculated for facet measurements taken between 2f and 2f + 4 m. These results are shown in Table 5.

Table 5 shows approximate 95% confidence error bands for SOFAST facet measurements taken between 2f and 2f + 4 m. Despite quoting ±3σ error bands (>99% confidence) for SOFAST outputs, 95% confidence intervals are stated to remain conservative given the assumptions inherent to the linear uncertainty model used. Facet focal length measurements are largely insensitive to setup errors. Twist measurements, largely due to the low magnitude of their nominal value are an order of magnitude more sensitive in a percentage sense. In an absolute sense, twist uncertainties are also very small. Global slope error is also quite insensitive to setup error as this term represents the RMS of the residual for all points on the facet. Local slope errors were relatively more sensitive as they were highly affected by target pixel mapping errors.

The SOFAST facet characterization system has a number of system hardware and setup inputs that must be measured/quantified prior to system operation. Uncertainties in these inputs translate to SOFAST measurement output uncertainties. A sensitivity and linear uncertainty analysis has shown that with reasonably careful setup, SOFAST can provide facet characterization with extremely low focal length uncertainty (<0.5%), twist measurements accurate to within ±0.03 mrad/m, local slope measurements accurate to within ±0.05 mrad, and global slope measurements accurate to within ±0.005 mrad. This analysis assumes that all output/input sensitivities are linear over the input distribution bounds, all inputs are linearly independent, and all inputs are uniformly distributed across a range derived from an analysis of the SOFAST system.

Results shown above demonstrate that linearly approximating output/input sensitivities is accurate. Probability distributions for input parameters are unknown so assuming uniform distributions ranging across conservative bounds is the proper approach. Refining input parameter distributions with normal distributions, if justified, would likely reduce uncertainties. Inputs were assumed to be independent as most inputs are stand-alone measurements that should display random errors. However, some inputs, such as the intrinsic camera parameters are clearly linked by the manner in which they are derived.

This manuscript has been authored by Sandia Corporation under Contract No. DE-AC04-94AL85000 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a nonexclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes.