ML25071A181
| ML25071A181 | |
| Person / Time | |
|---|---|
| Site: | 99902075 |
| Issue date: | 03/14/2025 |
| From: | Kerri Kavanagh Office of Nuclear Reactor Regulation |
| To: | Morton G Analysis & Measurement Services Corp |
| References | |
| EPID I-2025-201-0044 IR 2025201 | |
| Download: ML25071A181 (1) | |
Text
Mr. Greg Morton Chief Technical Officer and Quality Assurance Manager Analysis and Measurement Services 9119 Cross Park Drive Knoxville, TN 37923
SUBJECT:
NUCLEAR REGULATORY COMMISSION INSPECTION REPORT OF ANALYSIS AND MEASUREMENT SERVICES NO. 99902075/2025-201
Dear Mr. Morton:
On January 27 - 30, 2025, the U.S. Nuclear Regulatory Commission (NRC) staff conducted an inspection at the Analysis and Measurement Services (hereafter referred to as AMS) facility in Knoxville, TN. The purpose of this limited-scope routine inspection was to assess AMS compliance with provisions of Title 10 of the Code of Federal Regulations (10 CFR) Part 21, Reporting of Defects and Noncompliance, and selected portions of Appendix B, Quality Assurance Program Criteria for Nuclear Power Plants and Fuel Reprocessing Plants, to 10 CFR Part 50, Domestic Licensing of Production and Utilization Facilities.
This technically-focused inspection specifically evaluated AMS Online Monitoring Technology (OLM) software and services for NRCs regulated facilities. The enclosed report presents the results of the inspection. This NRC inspection report does not constitute NRCs endorsement of AMS overall quality assurance (QA) or 10 CFR Part 21 programs.
Within the scope of this inspection, no violations or nonconformances were identified.
In accordance with 10 CFR 2.390, Public inspections, exemptions, requests for withholding, and the NRCs Rule of Practice, a copy of this letter, its enclosure(s), and your response will be made available electronically for public inspection in the NRCs Public Document Room or from the NRCs document system (ADAMS), accessible at http://www.nrc.gov/reading-rm/adams.html.
Enclosure:
Inspection Report No. 99902075/2025-201 and Attachment March 14, 2025
G. Morton 2
Sincerely, Kerri Kavanagh, Chief Quality Assurance Vendor Inspection Branch Division of Reactor Oversight Office of Nuclear Reactor Regulation Docket No.: 99902075 EPID No.: I-2025-201-0044 Signed by Kavanagh, Kerri on 03/14/25
ML24270A144 NRR-106 OFFICE NRR/DRO/IQVB NRR/DRO/IQVB NRR/DEX/ELTB NRR/DRO/IQVB NAME AArmstrong DZhang DRahn KKavanagh DATE 3/12/2025 3/13/2025 3/14/2025 3/14/2025
Enclosure U.S. NUCLEAR REGULATORY COMMISSION OFFICE OF NUCLEAR REACTOR REGULATION DIVISION OF REACTOR OVERSIGHT VENDOR INSPECTION REPORT Docket No.:
99902075 Report No.:
99902075/2025-201 Vendor:
Analysis and Measurement Services 9119 Cross Park Drive Knoxville, TN 37923 Vendor
Contact:
Mr. Greg Morton Chief Technical Officer and Quality Assurance Manager Phone: (865) 691-1756 Ext. 118 Email: greg@ams-corp.com Nuclear Industry Activity:
Analysis and Measurement Services (hereafter referred to as AMS) scope of supply includes design, fabrication, calibration and testing of sensors and detectors, control rod drive motor testing and noise analysis, testing of plant protection system circuit boards, electro-magnetic compatibility, and cables testing, and field services and technical training for NRCs regulated facilities. AMS holds an accreditation from the American Association for Laboratory Accreditation as recognized by the International Laboratory Accreditation Cooperations Mutual Recognition Arrangement.
Inspection Dates:
January 27 - 30, 2025 Inspectors:
Aaron Armstrong NRR/DRO/IQVB, Team Leader Deanna Zhang NRR/DRO/IQVB, David Rahn NRR/DEX/ELTB Approved by:
Kerri Kavanagh, Chief Quality Assurance and Vendor Inspection Branch Division of Reactor Oversight Office of Nuclear Reactor Regulation
2 EXECUTIVE
SUMMARY
Analysis and Measurement Services 99902075/2025-201 The U.S. Nuclear Regulatory Commission (NRC) staff conducted a limited-scope routine vendor inspection at the Analysis and Measurement Services (hereafter referred to as AMS) facility in Knoxville, TN, to verify it had implemented an adequate quality assurance (QA) program that complies with the requirements of Appendix B, Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants, to Title 10 of the Code of Federal Regulations (10 CFR)
Part 50, Domestic Licensing of Production and Utilization Facilities, and 10 CFR Part 21, Reporting of Defects and Noncompliance. The NRC inspection team conducted this inspection on-site from January 27 - 30, 2025. This was the second NRC inspection at this facility.
This technically-focused inspection specifically evaluated AMS implementation of the quality activities associated Online Monitoring Technology (OLM) software and services for NRCs regulated facilities.
The following regulations served as the bases for the NRC inspection:
Appendix B to 10 CFR Part 50 10 CFR Part 21 During this inspection, the NRC inspection team implemented Inspection Procedure (IP) 43002, Routine Inspections of Nuclear Vendors, dated February 10, 2023; and IP 36100, Inspection of 10 CFR Part 21 and Programs for Reporting of Defects and Noncompliance, dated February 10, 2023.
The NRC inspection team observed the following specific activities:
Development and testing process for the Online Monitoring (OLM) data acquisition and analysis software Demonstration of Calibration Reduction System (CRS)21 software for analyzing acquired instrument data The results of the inspection are summarized below.
Inspection Areas The NRC inspection team determined that AMS established its programs for design control, test control, and nonconforming materials, parts, or components, in accordance with the applicable regulatory requirements of Appendix B to 10 CFR Part 50. Based on the limited sample of documents reviewed and activities observed, the NRC inspection team also determined that AMS is implementing its policies and procedures associated with these programs. AMS software products used for OLM data acquisition, qualification, and analysis were developed and tested using a documented software verification and validation (V&V) and Appendix B to 10 CFR Part 50 program. No findings of significance were identified in these areas.
3 REPORT DETAILS 1.
Design Control and Test Control a) Inspection Scope The Nuclear Regulatory Commission (NRC) inspection team reviewed Analysis and Measurement Services (hereafter referred to as AMS) policies and procedures that govern the implementation of its design control and test control programs to verify compliance with the regulatory requirements of Criterion III, Design Control, and Criterion XI, Test Control, of Appendix B, Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants, to Title 10 of the Code of Federal Regulation (10 CFR) Part 50, Domestic Licensing of Production and Utilization Facilities.
The NRC inspection team focused its inspection on the software design, development, and testing activities for Online Monitoring (OLM) software used to support NRC operating reactor licensees technical specification changes to switch from time-based monitoring calibration frequency for transmitters to a condition-based calibration frequency based on OLM results. The NRC inspection team also reviewed a sample of purchase orders from licensees of operating reactors for the procurement of OLM services.
The NRC inspection team reviewed AMS procedure for developing and modifying safety-related software and verified that the procedure documented an adequate process for initiating software changes or creation of new software, developing software requirements specifications (SRS) for the software, translating the software requirements into software design description (SDD), developing the software verification and validation plan (SVVP), and documenting the software verification and validation report (SVVR). The NRC inspection team also verified that development of software is documented such that (1) design activities are described in written procedures, (2) trained and qualified personnel are assigned to the activities, and (3) the design activities are reviewed and approved according to written procedures. The NRC inspection team verified that procedures for design and modification of software include data sheets that provide a format for assembling the documentation, including design and development notes, records of test results, design drawings, users guides, flowcharts, etc., and that SVVPs and SVVRs are completed, reviewed, approved, and included with the other documentation, which is controlled according to AMS procedures.
The NRC inspection team reviewed a sample of completed software development packages, including for BRIDGE24, Calibration Reduction System (CRS)21, and Instrument Performance Analysis Software System (IPASS)40 and verified that the sampled documents reflect the activities prescribed in the software development and modification procedure, as described below:
BRIDGE24 The NRC inspection team reviewed the initial software request (ISR) for modifying the BRIDGE software from version 23 to version 24 and verified that the software request adequately captured the reason for the software change. The NRC inspection team reviewed the SRS for BRIDGE24 and verified that it adequately captured the
4 functional and performance requirements, design constraints, and design attributes.
The NRC inspection team also reviewed the SDD for BRIDGE24 and verified that it adequately captured the design features, overall software architecture, expected outputs, and assumptions/limitations. The NRC inspection team reviewed the SVVP and SVVR for BRIDGE24 and verified that (1) the SVVP contained detailed steps to provide coverage of BRIDGE24 functions; and (2) the SVVR identified limitations and test anomalies and deficiencies, and the adequate disposition of these limitations and test anomalies and deficiencies. The NRC inspection team also verified that the tester was independent of the development of the BRIDGE24 software.
CRS21 The NRC inspection team reviewed the ISR for modifying the CRS software from version 20 to version 21 and verified that the software request adequately captured the reason for the software change, specifically to add additional software functionality and fix bugs. The NRC inspection team reviewed the SRS for CRS21 and verified that it adequately captured the functional and performance requirements, design constraints, and design attributes, and the inspection team observed that the SRS provided unique identifiers for each new feature and bug fix. The NRC inspection team also reviewed the SDD for CRS21 and verified that it adequately (1) captured the design features, overall software architecture, expected outputs, and assumptions/limitations, and (2) provided design descriptions that traced to the unique identifier of each new feature and bug. The NRC inspection team reviewed the SVVP and SVVR for CRS21 and verified that (1) the SVVP contained detailed steps to provide coverage of CRS21 functions that traced to the unique identifier of each new feature and bug; and (2) the SVVR identified limitations and test anomalies and deficiencies, and the adequate disposition of these limitations and test anomalies and deficiencies. The NRC inspection team reviewed the Module (Mod) QA SVVP and SVVR to address 7 deficiencies identified in the SVVR for CRS21 and verified that the software changes and retesting completed adequately addressed the 7 deficiencies.
The NRC inspection team also verified that the IV&V tester team was independent of the development team for the CRS21 software.
IPASS40 The NRC inspection team reviewed the ISR for modifying the IPASS software from version 30 to version 40 and verified that the software request adequately captured the reason for the software change, specifically to upgrade the software. The NRC inspection team reviewed the SRS for IPASS40 and verified that it adequately captured the functional and performance requirements, design constraints, and design attributes, and the inspection team observed that the SRS provided unique requirement identifiers. The NRC inspection team also reviewed the SDD for IPASS40 and verified that it adequately (1) captured the design features, overall software architecture, expected outputs, and assumptions/limitations, and (2) provided design descriptions that traced to the unique requirement identifier in the SRS. The NRC inspection team reviewed the SVVP and SVVR for IPASS40 and verified that (1) the SVVP contained detailed steps to provide coverage of IPASS40; and (2) the SVVR identified limitations and test anomalies and deficiencies, and the adequate disposition of these limitations and test anomalies and deficiencies, including the creation of a Mod QA package to address 6 of 7 the deficiencies and 1 of 4 limitations with remaining deficiency and limitations to be addressed in future revisions of IPASS.
5 The NRC inspection team also discussed the design control and test control program with AMS management and technical staff. The attachment to this inspection report lists the documents reviewed and personnel interviewed by the NRC inspection team.
b) Observations and Findings No findings of significance were identified.
c)
Conclusion The NRC inspection team concluded that AMS is implementing its design control and test control program in accordance with the regulatory requirements of Criterion III and Criterion XI of Appendix B to 10 CFR Part 50. Based on the limited sample of documents reviewed, the NRC inspection team also determined that AMS is implementing its policies and procedures associated with its design control and test control program. No findings of significance were identified.
2.
Nonconforming Materials, Parts, or Components a) Inspection Scope The NRC inspection team reviewed AMS policies and implementing procedures that govern the implementation of its nonconforming materials, parts, or components program to verify compliance with the regulatory requirements of Criterion XV, Nonconforming Materials, Parts, or Components, of Appendix B, to 10 CFR Part 50.
The NRC inspection team noted that AMS applies its nonconforming materials, parts, or components program to hardware failures. However, the NRC inspection team recognizes that software errors can be a form of nonconformance. As such, the NRC inspection team reviewed AMS processes and procedures that address software errors to verify that software errors are adequately identified and dispositioned. The NRC inspection team verified that AMS procedure for computer software and data control includes provisions to create software condition notifications (SCN) to address programming errors in software that have already been released. The NRC inspection team also verified that the SCN form requires an evaluation for 10 CFR Part 21 applicability. The NRC inspection team reviewed one SCN form completed for BRIDGE23 and verified that the programming error was adequately dispositioned. The NRC inspection team also reviewed the list of software bugs that were to be fixed in CRS21 and observed that these bugs were not adequately described in AMS software processes and procedures as described in Section 2.b below.
The NRC inspection team discussed the nonconforming materials, parts, or components program with AMS management and technical staff. The attachment to this inspection report lists the documents reviewed and personnel interviewed by the NRC inspection team.
b) Observations and Findings The NRC inspection team observed that AMS procedure for computer software and data control did not include provisions to address software bugs. The NRC inspection team held discussions with AMS staff to understand whether AMS procedures describe how software bugs listed in the SRS for CRS21 are documented, evaluated (including 10 CFR Part 21 applicability) and dispositioned. AMS staff clarified that the while AMS procedures does not
6 describe how bugs are dispositioned, their definition of bugs are those of low safety-significance and would not rise to a level of needing a 10 CFR Part 21 evaluation. The NRC inspection team selected a sample of the bugs listed in the SRS for CRS21 and verified that these bugs were of low safety-significance, and they had no effects on the results of the OLM analysis. As such, the NRC inspection team determined that this issue is minor.
AMS created Corrective Action Report (CAR) 25-01-01 to address this issue.
No findings of significance were identified.
c.
Conclusion With the exception of the minor issue noted above, the NRC inspection team concluded that AMS is implementing its nonconforming materials, parts, or components and corrective action programs in accordance with the regulatory requirements of Criterion XV of Appendix B to 10 CFR Part 50. Based on the limited sample of documents reviewed, the NRC inspection team also determined that AMS is implementing its policies and procedures associated with its nonconforming materials, parts, or components program. No findings of significance were identified.
3.
Entrance and Exit Meetings On January 27, 2025, the NRC inspection team discussed the scope of the inspection during the entrance meeting with Dr. Hash Hashemian, AMS President and CEO, and other members of AMS management and technical staff. On January 30, 2025, the NRC inspection team presented the inspection results during an exit meeting with Mr. Greg Morton and other members of AMS management and technical staff. The attachment to this report lists the attendees of the entrance and exit meetings, as well as those individuals whom the NRC inspection team interviewed.
Att1-1 ATTACHMENT 1.
ENTRANCE/EXIT MEETING ATTENDEES Name Position Affiliation Entrance Exit Interviewed Dr. Hash Hashemian President and CEO Analysis &
Measurement Services (AMS)
X X
Greg Morton Chief Technological Officer and Quality Assurance (QA)
Manager AMS X
X X
Chad Kiger Electro Magnetic Compatibility (EMC)
Engineering Manager AMS X
X X
Eternity Perry Assistant QA Manager AMS X
X X
Allyson Pearce QA Consultant AMS X
Sam Caylor Systems Development Manager AMS X
Chris Maddux Senior Software Developer AMS X
Brent Shumaker Senior Engineering Manager AMS X
Darrell Mitchell Technical Services Manager AMS X
Ryan OHagan Marketing Manager AMS X
Adam Deatherage Applications Engineer AMS X
Att1-2 Aaron Armstrong Inspection Team Leader Nuclear Regulatory Commission (NRC)
X X
Denna Zhang Inspector NRC X
X David Rahn Inspector NRC X
X Kerri Kavanagh Branch Chief NRC X*
- Remote 2.
INSPECTION PROCEDURES USED:
Inspection Procedure (IP) 43002, Routine Inspections of Nuclear Vendors, dated February 10, 2023 IP 36100, Inspection of 10 CFR Part 21 and Programs for Reporting of Defects and Noncompliance, dated February 10, 2023 3.
DOCUMENTS REVIEWED Quality Assurance Procedures (QAP)
QAM0101, Quality Assurance Manual, Revision 12, dated January 2021 CAT9201, Procedure for Producing and Tracking Condition Reports of Corrective Action, Revision 9, dated February 2020 PDC8701, Procedure for Design Control, Revision 0, dated February 1987 SCM0101, Software Configuration Management Plan for Analysis and Measurement Services Corporation, Revision 4, dated May 2014 SDC8601, Procedure for Computer Software and Data Control, Revision 10, dated August 2016 SDM9201, Procedure for Software Development or Modification, Revision 8, dated August 2016 TCR8901, Administration of 10 CFR Part 21 Requirements, Revision 8, dated March 2020 Design Control (Software Package)
Initial Software Request (ISR) for IPASS40, dated January 5, 2023 Software Requirements Specification (SRS) for IPASS40, Revision 1, dated January 5, 2023 Software Design Description (SDD) for IPASS40, dated August 17, 2023 Software Verification and Validation Plan (SVVP) for IPASS40, dated August 23, 2023 Software Verification and Validation Report (SVVR) for IPASS40, dated August 25, 2023 SVVR Cover Sheet IPASS40, dated August 25, 2023 Software Verification and Validation Modular Test Plan for IPASS40 V.4.0.0, dated August 25, 2023 SVVR for IPASS40 Modular QA, dated August 25, 2023 SVVR Cover Sheet for IPASS40 Modular QA, dated August 25, 2023
Att1-3 Program Listing for IPASS40, dated August 23, 2023 SVVR for IPASS40 v4.0.01.38, dated July 8, 2024 Software Verification and Validation Modular Test Plan for IPASS401, dated June 25, 2024 SDD for IPASS40 v4.0.1, Modular Changes, dated June 25, 2024 SVVR Cover Sheet for IPASS401 Mod, dated July 8, 2024 Software Release Documentation (SRD) for IPASS40, v4.0.1, dated July 9, 2024 Program Listing for IPASS40, v4.0.1, dated July 9, 2024 ISR for CRS21, dated April 5, 2024 SRS for CRS21, dated April 5, 2024 SDD for CRS21, dated April 5, 2024 SVVP for CRS21, dated April 5, 2024 SVVR for CRS21, dated April 26, 2024 SVVR for CRS21 v2.1.1.0 (Module QA), dated April 29, 2024 Program Listing for CRS21, dated June 28, 2024 SRD for CRS21 v2.1.1.1, dated April 29, 2024 SVVR for CRS 20, dated June 2, 2023 ISR for BRIDGE24, dated November 28, 2018 SRS for BRIDGE24, dated November 28, 2018 SDD for BRIDGE24, dated December 11, 2018 SVVP for BRIDGE24, dated December 18, 2018 SVVR for BRIDGE24, dated December 20, 2018 SVVR Cover Sheet for BRIDGE24, dated December 18, 2018 Program Listing for BRIDGE24, dated January 4, 2019 Purchase Orders (PO) 10341625, dated June 26, 2024 10341154, dated June 18, 2024 7566998, dated December 8, 2023 3501378628, dated September 12, 2024 10346252, dated September 30, 2024 10316085, dated July 17, 2023 Software Condition Notification Software Condition Notification 140501, dated May 22, 2014 Corrective Action Reports Opened During the Inspection CAR 25-01-01, dated January 29, 2025 Training Records Software Development Training Record for Elizabeth Liberacki, dated April 3, 2018 Software Development Training Record for Terrell Mclllwain, dated January 10, 2023
Att1-4 Software Development Training Record for Sam Caylor, dated August 24, 2016 Software Development Training Record for Zachary Becker, dated June 22, 2022 Software Development Training Record for Chris Maddux, dated August 24, 2016 Qualification Certificate for Terrell Mclllwain as Level II Staff Engineer, dated February 2024 Qualification Certificate for Zachary Becker as Level I Staff Engineer, dated February 2024 OLM Analysis Reports Results of End of Cycle 22 Analysis of On-Line Calibration Monitoring Daa for Pressure Transmitter for PO 10316085, dated September 2023 Checklist for transmitter on Online Calibration Report, dated September 13, 2023