GD&T Inspection: Method

Position, Bonus and Datum Shift; Least-Squares and Minimum-Zone Form Fits; Runout

What the tool does

A geometric tolerance on a drawing is a statement about a zone: an axis must lie inside a cylinder of a stated diameter about its true position, a surface must lie between two parallel planes a stated distance apart, a profile must lie between two concentric circles. Inspection reduces measured coordinates to the one number that is compared with the zone, and the rules for doing so are what this tool implements. The position part takes a table of measured features and applies the arithmetic of ASME Y14.5: the diametral deviation, the bonus from the actual size, the datum shift of the pattern, the composite feature-relating check and the projected zone. The form parts take pasted points and fit the reference geometry two ways, by least squares (the algorithms NIST wrote as reference software for coordinate measuring machines) and by the minimum zone that the standard actually defines, because the two can disagree on a verdict. The orientation and runout parts follow the same pattern. The last part prints the boundaries of a feature of size and the equivalence between coordinate and position tolerances.

Position of a feature

A position tolerance Ø T defines a cylindrical zone of that diameter whose axis is the true position, located by basic dimensions from the datum reference frame. The measured axis of the feature (the axis of its unrelated actual mating envelope) is at (x_m, y_m), the true position at (x_n, y_n), and the smallest zone that contains the axis has the diameter d = 2√(Δx² + Δy²) with Δx = x_m − x_n, Δy = y_m − y_n. The factor 2 is the reason a position tolerance of Ø 0.010 allows only a 0.005 radial error, and the reason a ± 0.005 coordinate tolerance is not the same thing (see the conversions below). The feature accepts when d ≤ allowable, where the allowable is T plus the bonus.

Bonus tolerance. When the feature control frame carries the MMC modifier, the stated tolerance applies at the maximum material condition and grows by the amount the actual size departs from it: bonus = |actual size − MMC size|. For a hole MMC is the smallest size, so a larger hole earns bonus; for a pin MMC is the largest size. With the LMC modifier the bonus is the departure from the least material size, and regardless of feature size (RFS) there is none. The actual size used is the actual mating size, the largest perfect pin that fits the hole or the smallest ring that passes over the pin, not a two-point diameter. A feature whose size is outside its limits is rejected on size whatever its position; the tool caps its bonus at the full size tolerance so that the position margin printed is still meaningful.

Datum shift. When a datum feature of size is referenced at its maximum material boundary (MMB), the datum feature simulator is a fixed boundary of that size, and a datum feature made away from it has clearance to the simulator. That clearance is the datum shift: the part can move relative to the datum reference frame, and so the whole pattern of features can move with it, by up to half the clearance in any direction. The shift is not a bonus added to each feature, because the pattern must move as one; every feature must pass simultaneously at a single position of the pattern. The tool searches the translation inside the allowed circle that minimises the worst margin over the features and reports the pattern verdict at that position. A rotation of the part about the datum axis, which a single datum pin would also allow, is not credited. The clearance is entered as a diameter (the datum feature's actual mating size minus its boundary size); the tool does not know the datum feature's own tolerance.

Composite position. A composite feature control frame has two segments with one position symbol. The upper segment, the pattern-locating tolerance zone framework (PLTZF), locates the pattern to the datums named and is the feature table of the tool. The lower segment, the feature-relating tolerance zone framework (FRTZF), has a tighter tolerance and holds the features at their basic spacing to each other; with only the primary datum repeated it is free to translate and rotate as a whole (it keeps only the orientation to that datum). The check is therefore a best fit: the tool finds the translation and rotation of the pattern of zones that minimises the worst margin over the features, and the pattern accepts when every feature then lies in its feature-relating zone. The bonus from the actual size applies to the lower segment as well when it carries the modifier.

Projected tolerance zone. When a mating part's clearance holes must clear a threaded or press-fit pin whose axis tilts within the hole, the position zone is projected above the surface by the mating part's thickness (the P modifier). The tool extends the measured axis through its entry point and its exit point, a depth t_d below the surface, to the projected height h_p: the deviation there is Δ + (Δ − Δ_e) h_p/t_d, and the axis must stay inside the zone over the whole projected height, so the larger of the deviation at the surface and at the projected height is compared with the allowable. The tilt angle is reported as well.

Feature-of-size boundaries and the coordinate equivalence

The virtual condition is the constant boundary generated by the collective effects of the specified material condition and the geometric tolerance: for an internal feature at MMC it is MMC − T, for an external feature MMC + T; it is the boundary a mating part must clear, and the size of the functional gauge pin or hole. The resultant condition is the worst-case boundary at the opposite limit of size with the tolerance and the full bonus: for a hole at MMC it is LMC + T + (LMC − MMC). The inner and outer boundaries are the smallest and largest envelopes the feature can present over its range of size and position. With the LMC modifier the roles reverse, and regardless of feature size there is no fixed virtual condition, only the two boundaries.

A ± t coordinate tolerance in x and y is a square zone 2t on a side. The circle that circumscribes the square has the diameter 2√2 t, so a position tolerance of that diameter accepts every part the coordinate tolerance accepts plus the corners it rejects, π/2 = 1.57 times the area; the usual statement is that position gives 57 % more tolerance. Conversely the square inscribed in a position zone of diameter T is ± T/(2√2) = ± 0.354 T, the coordinate tolerance that guarantees the position. The bonus chart draws the allowable position tolerance against the actual size from LMC to MMC, with each measured feature plotted at its size.

Least-squares fits

Every form analysis starts with the least-squares (Gaussian) fit, the fit that minimises the sum of the squared orthogonal distances from the points to the geometry. The algorithms are those of Shakarji's reference software for the NIST Algorithm Testing System (J. Res. NIST 103, 633, 1998). For a plane and a line the centroid lies on the fit and the orientation is an eigenvector of the 3 × 3 scatter matrix of the centred points: the smallest eigenvalue's for the plane normal, the largest's for the line direction; the tool solves the eigenproblem by Jacobi rotations and refines the plane normal by one linear fit in the frame of the first estimate, because forming the scatter matrix squares the condition number of the data. For circles, spheres and cylinders the distance to the geometry is nonlinear in the parameters and the sum of squares is minimised by the Levenberg–Marquardt method with the distance derivatives the paper lists, the parameters normalised at every step (unit axis direction, axis point nearest the centroid). The circle is started from Pratt's algebraic fit, which is itself printed beside Kåsa's simpler algebraic fit; the cylinder is restarted from the three principal directions of the points and the three coordinate axes and the lowest sum of squares is kept, the paper's technique for short cylinders and small patches. A three-dimensional circle is fitted by projecting the points onto their least-squares plane, which is also how the NIST reference fits were made.

Every fit was checked against the 240 NIST L2 reference pairs, data sets generated under ASME B89.4.10 with their least-squares fits computed to 60 digits, which NIST distributes for self-testing: the plane normals, line directions, circle centres and diameters, sphere centres and cylinder axes agree with the reference to better than 10⁻⁷ mm on parts of up to 500 mm placed up to 1,000 mm from the origin.

Minimum-zone fits

Y14.5 defines flatness as the distance between two parallel planes that contain the surface and are as close together as possible, and the other form tolerances in the same words: two parallel lines, two concentric circles, two coaxial cylinders. That is a Chebyshev (minimax) fit, not a least-squares one, and the two give different numbers: the least-squares spread (the largest minus the smallest distance about the least-squares fit) is never smaller than the minimum zone and is usually a few per cent larger. A surface can fail by the least-squares spread and pass by the minimum zone; the standard's number is the minimum zone, and the tool prints both so the difference is visible. Shakarji (2004) showed that the objective functions of these fits are not smooth and have several local minima near the global one, so a naive downhill search from the least-squares fit can stop at the wrong answer; his reference algorithms use simulated annealing. This tool uses exact or exhaustive methods where they exist and multi-start local searches elsewhere.

Flatness and straightness. Write the candidate plane as z = a x + b y + c in a frame whose z axis is the least-squares normal. The vertical zone W(a, b) = max(z_i − a x_i − b y_i) − min(z_i − a x_i − b y_i) is a convex, piecewise-linear function of the slopes, so its minimum is found exactly by a nested golden-section search (the partial minimum over b of a jointly convex function is convex in a). The flatness is the perpendicular distance between the two planes, W/√(1 + a² + b²), which is then polished directly. The same construction in one variable gives the straightness of a line of points; the straightness of a measured axis (three-dimensional points) is the diameter of the smallest cylinder about a line that contains them, found by a multi-start Nelder–Mead search over the line's piercing point and direction in the least-squares frame. The result is the same as the Chebyshev linear program and was checked against it.

Roundness. Four reference circles are printed, as ISO 12181 names them. The minimum-zone circle is the centre that minimises max r_i − min r_i; the tool evaluates that objective on a grid of starts about the least-squares centre and refines the best of them by Nelder–Mead, which reproduces a brute-force grid search on the NIST circle sets. The minimum circumscribed circle, the smallest circle containing every point, is found exactly by Welzl's move-to-front algorithm. The maximum inscribed circle follows Shakarji's definition: no point inside, touching three points that form an acute or right triangle, the largest such; up to 120 points it is found exactly by enumerating the point triples (the centre is a vertex of the Voronoi diagram of the points), and for larger sets by a multi-start local search, which the page then says. A data set can have two maximum inscribed circles of the same radius (the paper's two-lobed example reproduces both); the tool reports one. The roundness of each fit is the radial spread about its own centre; the minimum zone is the smallest and is the value to report. The roundness analysis takes a closed profile; a partial arc has no inscribed circle in this sense.

Cylindricity. In the least-squares frame a nearby cylinder is described by the point where its axis pierces the x y plane and the two components of its direction (Shakarji 2004, Table 2); the zone max f_i − min f_i of the radial distances is minimised over those four parameters from several starts. The cylindricity is the zone, the radius its mid-value.

Orientation to a datum plane

The datum plane is the least-squares plane of the points measured on the datum feature. Y14.5 defines the datum by the high points of the feature (the plane a surface plate would touch); the least-squares plane is the common CMM substitute and differs from it by a fraction of the datum feature's flatness, which the tool prints. The tolerance zone is two parallel planes whose normal makes the basic angle α with the datum normal: 0° for parallelism, 90° for perpendicularity, the stated angle for angularity. Their azimuth about the datum normal is not specified by the drawing (a perpendicular face may lean in any direction), so the zone width, the spread of the feature points' heights on the candidate normal, is minimised over the azimuth by a 360-step scan refined by golden section. For parallelism the azimuth does not enter and the zone is simply the spread of the heights above the datum plane. The feature's own least-squares plane gives its actual angle to the datum for the report, and its flatness, which is a lower bound on any orientation zone.

Runout

Circular runout is the full indicator movement during one revolution about the datum axis at one section, max − min of the readings; total runout is the full movement over the whole surface, max − min over every section pasted. A first-harmonic fit r = a₀ + e cos(θ − φ) by least squares separates the eccentricity of the section from the datum axis, whose own runout is 2e, from the residual form, the spread of the readings after the harmonic is removed; the two tell the machinist whether to move the part or to improve its form.

Entering data

The position table takes the true position, the measured axis at the entry surface and the actual size of each feature, with optional exit coordinates for the projected zone; lengths go through the unit selector (in or mm). The pasted boxes take one point per line, x y or x y z, separated by spaces, commas or tabs, in any one unit; a line beginning with # names a section (used by the runout box). Their results are reported in the unit pasted, and the tolerance for the verdict is entered in that unit. The default data are illustrative: a four-hole pattern with one feature out of position, a 4 × 3 flatness grid, a 24-point roundness profile with two- and three-lobe form and a centre offset, a 24-point cylinder with a tilted axis, a datum face and a perpendicular face, and two runout sections. The cylinder example for the cylindricity option is the following set (x y z, inches): 1.49976 1.50000 0.00015; 1.64644 1.14644 0.00004; 2.00000 1.00025 −0.00010; 2.35360 1.14640 −0.00018; 2.50016 1.50000 −0.00015; 2.35355 1.85355 −0.00004; 2.00000 1.99985 0.00010; 1.64649 1.85351 0.00018; 1.49986 1.49990 0.50015; 1.64656 1.14631 0.50004; 2.00015 1.00010 0.49990; 2.35378 1.14627 0.49982; 2.50036 1.49990 0.49985; 2.35374 1.85349 0.49996; 2.00015 1.99980 0.50010; 1.64660 1.85345 0.50018; 1.49996 1.49980 1.00015; 1.64667 1.14617 1.00004; 2.00030 0.99995 0.99990; 2.35397 1.14613 0.99982; 2.50056 1.49980 0.99985; 2.35392 1.85342 0.99996; 2.00030 1.99975 1.00010; 1.64672 1.85338 1.00018 (paste one point per line).

Limits

The position analysis is two-dimensional (axes normal to the datum plane; the projected zone adds the tilt); the measured axis location must come from the measuring machine's own mating-envelope fit. The datum shift is a translation only and the composite fit keeps the orientation to the primary datum; a lower segment that repeats secondary datums would also fix the translation in one direction, which the tool does not model. The orientation datum is the least-squares plane, not the high-point plane. The maximum inscribed circle is exact up to 120 points, the minimum-zone circle and cylinder are multi-start local searches checked against brute force on the NIST sets, and all form values are for the points given: a sparse sample underestimates the form error of the surface. Profile tolerances, concentricity and symmetry, and datum reference frames built from several datum features are not computed.