Precision machining: tolerances, surface finishes and dimensional control
What defines precision machining?
Precision machining is the ability to manufacture parts within tight dimensional and geometric tolerances, with controlled surface finishes and documented verification of every dimension. It is not simply a matter of whether a CNC machine can achieve a given tolerance — it is about guaranteeing that tolerance repeatably, documenting it and maintaining it part after part.
In practice, industrial precision machining works with tolerances ranging from ±0.025 mm (IT7–IT8 grades, the standard reference in CNC manufacturing) down to ±0.005 mm for specialised applications such as aerospace components or instrumentation. These figures demand rigorous process control: thermal stability, fixture rigidity, tool quality and three-dimensional measurement.
At MECVIL, precision machining is a cross-cutting capability that we apply to CNC milling, CNC turning, grinding and 5-axis machining alike. Our ISO 9001 quality system establishes the inspection and traceability protocols that guarantee every specified tolerance.
Tolerances according to ISO 2768: the industrial drawing reference
ISO 2768 defines the general tolerances applied when a drawing does not specify a particular tolerance for each dimension. It is the most widely used reference on industrial drawings and is divided into two parts:
ISO 2768-1: linear and angular tolerances
| Dimensional range | Fine (f) | Medium (m) | Coarse (c) |
|---|---|---|---|
| 0.5 – 6 mm | ±0.05 mm | ±0.1 mm | ±0.3 mm |
| 6 – 30 mm | ±0.1 mm | ±0.2 mm | ±0.5 mm |
| 30 – 120 mm | ±0.15 mm | ±0.3 mm | ±0.8 mm |
| 120 – 400 mm | ±0.2 mm | ±0.5 mm | ±1.2 mm |
| 400 – 1,000 mm | ±0.3 mm | ±0.8 mm | ±2.0 mm |
| 1,000 – 2,000 mm | ±0.5 mm | ±1.2 mm | ±3.0 mm |
ISO 2768-2: geometric tolerances
This part defines three classes (H, K, L) for flatness, perpendicularity, symmetry and concentricity. The most common combination on industrial drawings is ISO 2768-mK (medium linear tolerance + geometric tolerance K).
For parts with requirements exceeding ISO 2768, tolerances are specified individually dimension by dimension, applying ISO 286 (shaft–hole fit system) and ISO 1101 (GD&T geometric tolerances) with its 14 control types: flatness, cylindricity, roundness, perpendicularity, concentricity, position, circular and total run-out, among others.
Need to machine parts with tight tolerances?
At MECVIL we work with tolerances from ±0.025 mm on CNC machining of parts up to 20 metres long. Request a quotation with your drawings and specifications.
Surface finish: Ra, Rz and N grades
Surface finish is measured according to ISO 4287 and is usually expressed as Ra (arithmetic mean roughness, in micrometres). Each CNC machining process delivers a different finish range:
| Process | Typical Ra | N grade | Application |
|---|---|---|---|
| Roughing (milling/turning) | 3.2 – 6.3 µm | N8 – N9 | Non-functional surfaces |
| Standard CNC finish | 1.6 – 3.2 µm | N7 – N8 | General-purpose parts |
| Fine CNC finish | 0.8 – 1.6 µm | N6 – N7 | Bearing seats, guideways |
| Grinding | 0.1 – 0.8 µm | N4 – N6 | Sealing surfaces, calibrated shafts |
| Lapping/polishing | 0.05 – 0.4 µm | N3 – N5 | Optics, instrumentation |
It is important to specify roughness only on surfaces that functionally require it. Improving the finish from Ra 3.2 to Ra 0.8 µm can increase machining cost by 200 % to 400 %, as it requires additional passes with specialised tooling or secondary processes such as grinding.
How is precision controlled? Metrology and dimensional verification
Manufacturing a part to precision demands verification that it actually meets the specified tolerances. At MECVIL we use coordinate measuring machines (CMM) with 1–2 micrometre spatial accuracy in 3D measurement, integrated into our ISO 9001 quality system.
The dimensional control process follows these stages:
- 1.Raw material inspection: verification of dimensions and material certificates before machining begins
- 2.In-process control: measurement of critical dimensions during intermediate machining stages to detect deviations before they become irreversible
- 3.Final CMM inspection: complete three-dimensional measurement of all toleranced dimensions, with generation of a dimensional report
- 4.Documentation and traceability: each part is linked to its inspection report, material certificates and process record
Our final verification fixture project with dimensional control reporting is an example of how precision machining does not end at the last pass of the CNC machine, but at the documentation that accompanies it.
For applications requiring automated inspection, we integrate machine vision and 3D profilometry systems directly into production lines.
Precision machining of large-dimension parts
One of the most demanding technical challenges is maintaining precision on large-scale parts. When a component measures 3, 5 or up to 20 metres, standard tolerances must coexist with additional factors:
- Thermal deformation: expansion caused by ambient temperature and cutting heat affects long parts. Thermal compensation and measurement under controlled conditions are required.
- Deflection under self-weight: heavy parts can deform under their own weight during machining. Fixturing and support must be calculated to minimise this deflection.
- Machining strategy: the sequence of passes must be planned to release residual stresses in a controlled manner, preventing post-machining distortion.
At MECVIL we have fixed-bed milling machines with travels of up to 20 metres and three-dimensional measurement equipment to verify the geometry of large-format parts. Our article on large-dimension CNC machining explores this topic in greater detail.
Materials and their impact on precision
Each material behaves differently under cutting conditions, and this directly affects the achievable precision:
- Aluminium (6061, 7075): excellent dimensional stability, permits high speeds and fine finishes. Ideal for machining aeronautical and railway profiles.
- Carbon steel (C45, 42CrMo4): good machinability, predictable behaviour. The reference for tooling and machine beds.
- Stainless steel (304, 316L): work hardens during machining, which can alter geometry if cutting parameters are not controlled.
- Titanium (Ti-6Al-4V): low thermal conductivity concentrates heat at the cutting zone, generating distortion if not cooled adequately. Requires carbide tools with TiAlN coating and reduced speeds.
- Superalloys (Inconel 718): the greatest challenge in precision machining. High hardness, a tendency to work harden and accelerated tool wear.
The choice of cutting parameters, tooling and cooling strategy depends directly on the material. At MECVIL, more than 50 years of experience with these materials allow us to optimise every process for maximum precision at minimum cost.
Criteria for choosing a precision machining supplier
When selecting a manufacturing partner for high-precision parts, evaluate:
- Measurement equipment: does the supplier have CMM with certified traceability? Does it generate complete dimensional reports?
- Multi-axis capability: 5-axis machining reduces repositioning errors and improves geometric precision
- Certifications: ISO 9001 guarantees a quality system with traceability; EN 1090 certifies competence in metallic structures
- Sector experience: a workshop serving 13 sectors has the protocols to meet the demands of each industry
- Dimensional capacity: can it maintain tolerances on parts of every size your project requires?
At MECVIL we combine CNC machining of parts up to 20 metres, simultaneous 5-axis capability, precision grinding and three-dimensional measurement across more than 10,500 m² of facilities. Our 360° turnkey service includes the engineering that optimises the design for manufacturability, and the assembly that ensures part precision is transferred to the final assembly.
Looking for a manufacturer that can guarantee tight tolerances with full documentation?
At MECVIL we verify every part with three-dimensional measurement and deliver dimensional reports with ISO 9001 traceability. Contact our technical team to evaluate your project.