IR 05000275/1991004
| ML20059E512 | |
| Person / Time | |
|---|---|
| Site: | Diablo Canyon |
| Issue date: | 08/24/1993 |
| From: | Kirsch D, Miller L NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V) |
| To: | |
| References | |
| OLA-2-I-MFP-070, OLA-2-I-MFP-70, NUDOCS 9401110244 | |
| Download: ML20059E512 (11) | |
Text
..
..
-
.-.
.
..
.
-
_- -
-
-
__
t ~u e n
.mFP IDenM l#
- . & 2n/a'3 * s gggg gnes wgt '
l ; *.
U.S. NUCLEAR REGt/LATORY C0 MIS $10N i
gegg REGION Y dNHC
.
.
..
-~
j Report Nos.
50-275'/91-04 and 50-323/91-04
.
!
Docket Nos.
50-275 and 50-323
~
'
s.<,,
,w
/o
!
License Nos.
i Licensee:
Pacific Gas and Electric Company
77 Beale Street
.
-
San Francisco, California 94106.
Facility Mame:
Diablo Canyon Units 1 and 2 l
Inspection Conducted:
February 12-24, 1991 i
Inspector:
~
i
/
i L. uiller, chtef. Lperations section Date signea i
Approved by:
{
_
[" D /
D. W. Kirsch, Ch eff Reactor Safety Br{afich Date Signed.
i.
l Sumary:
Inspection on February 11-14,1991 (Report Nos. 50-275/91-04and50-523/91 i
i Areas insoected:
!
..
This was an announced, special inspection to follow-up on the inspection i
documented in Inspection Report Nos. 50-275/90-29 and 50-323/90-29 i
mechanical maintenance measuring and test equipment 091 MATE) issues at~concerning j
Diablo Lt. yon. Inspection proceuvres 30703 and 92700 were used.
i j
Results:
'
The inspection made the following general conclusions:
i
!
i I
3.
The licensee's management at the site and corporate offices had not ensured that the Quality Assurance (QA) and Quality Contro1~ (QC)
i
-
Departments' significant audit findings relating to the M87E program were
substantively followed up and corrected by the Maintenance Department.
j i
2.
j The QC Department, and to a lesser elegree, the QA Department, were not
'
aggressive in seeking corrective action for the findings of their audits j
of M MATE.
'.
^
i 9401110244 930824 l
PDR ADOCK 05000275 O
.
.
-
~
j f
<
'i ct40ut opyy
!
,
..
.-
.
-. -,
.-.-..
-
i
.,.
+
-
.
,
.
!
'
.
I
4. _ _
_3.
The methods and equipment used by-the-Mechanical Maintenance Department- ---- -
to control MATE were still inadequate, despite the previous. licensee
'
audits and hRC inspection. (50-275/90-29), based on a limited sample of j
activities.
j These findings appear to represent a significant safety matter because they, j
indicate a chronic progransnatic weakness in the control of M M&TE. which may i
have, or at least had the potential to adversely impact installed safety-
related equipment. Further, although these deficiencies in the control of
,
E M&TE were identified by the QA and QC organizations,. these oversight groups.
Mechanical Maintenance and PG&E management were ineffective in achieving the
.
j necesse y corrective action.
Oneapparentviolation(50-275/91-04-02) was identified: failure to identify a breakdown in the program for control of measuring and test equipment as a
j nonconformance, and failure to promptly correct this breakdown.
- -
{
Unresolved item 50-275/90-29-01 was resolved by this inspection into violation-
!
50-275/91-04-02, and is therefore closed.
Three open items were identified.
i i
i
!
,
!
1
,
..
!
!,
-.
...
~
- .,
Ii
,"
!
Q
yI
'
$O VI lf epi.
i
{
sl ; $; = ' _ s.
,
s
.%
,
i
=
$ ll'WI,
'
n a
e y *9- $
j
}-
0 N.
x%
.
'e
'
E t $,'
NiW A
p e
I
\\.' s n 8'
g
$u lc," E e
.
E i-
"
.
sx
<r
N Y
g
.
.. - --.-
_
_
_.
.
_
_
_
.
__
_ _ _ _
_. _
. _
,
i i!,-
!
!
t
DETAILS i-l 1.
persons Contacted
.
.. - - - -
- J. Townsend. Vice President. NPG Plant Manager
- W. Barkhuff. Quality Control Manager i
T. Bennett, Mechanical Maintenance Department Manager l
C. Seward. Senior power Production Engineer M. O'Connell. Regulatory Compliance Engineer
!~
D. Taggart. Director Site Quality Assurance
,
J. Strahl. Mechanical Maintenance Foreman
'
i
- B. Giffin Asst. plant Manager. Maintenance Services A. Young. Sr. QA Supervisor
- T. Grebel. Regulatory Compliance Supervisor
' Attended exit meeting.
'
.
The inspectors also held discussions with other licensee and contractor personnel during the inspection.
l 2.
Background
,
!
-
This inspectior ses a followup inspection to Inspection Nos. 50-275/g0-29-
"
l and 50-323/g0-29. The purposes of the latest inspection were threefold:
to review further the existing program for control of measuring and test
-
equipment (M4TE). to review the corrective actions which had been taken j
for deficiencies identified in earlier versions of that program by the i
licensee's audits and the previous NRC inspection, and to determine whether j
enforcement action was appropriate for the unresolved item identified by i
that inspection. This unresolved item concerned what appeared to be ineffective corrective action for the M&TE propram weaknesses previously
j identified by the licensee's audits'and survei lances.
,
3.
Review of Existing program for Control of M87E i
The inspector conducted surveillances of work by personnel-in the Mechanical Maintenance (191) and Instrumentation and Controls (!&C)
{
calibration and tool issuing facilities. Selected ~ tool issue logs, i
calibratica records. M8TE modules from the Plant Infomation Management
!
System (PIMS), and personnel qualification records were reviewed. In l
addition. tool issuance, return, and calibration were observed.
I a.
The inspector observed that calibrated tools were issued and j
calibrated by the 14C department in a careful and methodical
manner. Licensee representatives stated at the beginning of the
inspection that all work performed in the radiologically i.
controlled area (RCA) during the refueling outage in progress i
,
i would be done using only M8tE issued by the I&C department. This-
'
policy was announced in a memorandum dated November 21,1990,and l
i became effective' January 14,1991, shortly after the end of the i
i previous 4RC inspection.
!
$
l l
i i-
.
l I
i
-
--.-.
- -..
--
-
. - -.
. _ - -
. _ - -. - - _ _ _ - - - _
._
Z-
,
+
,
The inspector determined that because of tt change'in policy,
.
approximately 75% of the calibrated torque w.enches were under the
.-
control of the 14C department.:a higher percentage than was found
during the previous inspection..
'
The only potential discrepancy observed'in the ISC department control program was that calibration' personnel-stated that they did not have'
i an effective method to ensure that issued tools were promptly returned-
'
when the job to which the tools had been assigned had-been completed.
b.
The inspector observed that cailbrated tools were issued and calibrated by the M department in a manner which was generally
)
consistent with the less detailed procedur t1 guidance required by
'
the licensee for this department. However, the following i
significant discrepancies were identified:
(1) On February 12, 1991, the inspector observed that Tension ~
Dynamometer #157 had been issued for Work Order C0078894-01 t
on January 23, 1991, a work order which was completed on i
,
January 29, 1991. This was not a safety related job, but was associated with safely moving a cask used for transportation of radioactive material.
At the inspector's request.'the.
tool was located. The personn::1 using it stated that'it had:
been in use to check chainfalls for.some days, a different.'
job than the one for which it had been issued.
The licensee personnel were not aware of this earlier job.-
The inspector noted that this was a current example of'a finding.
..
.
identified in QC Surveillance Report QCS 1990, and a siellar one re-identified in Quality Assurance Audit 90-i 908127, dated September 6, 1990..The surveillance report had
'
stated that 285 of the M&TE usage sampled was not recorded as required by Quality Assurance procedure (QAP) 12.A. Control and Calibration of Measuring and Test Equipment, Sections 4.25'and '
4.26, Revision 19.
The QA audit stated that 35% of work orders by AP C-4053, Revision 12. Attachment 6.3. Instructi
,
i Completing and handling W/0% -ith p!MS on Line.-
j
"
..
Finally, M M&TE personnel stated that personnel frequently did
-
.
not return M&TE once the job for which it had been issued was completed, and that testing personnel did not have an effective method to ensure that the equipmeht was returned.
limited sample of one job which was checked, and the coments-Based on the licensee's controls to ensure that M8TE was t jcbs where it was used were still ineffective.
item (50-275/91-04).
This is an open
. -
. -.
._- -.
-
.
.
..-_
.
.
. - _ _ - _ - _. _ _ _ _ _ _ _ _
_
.,
i.-
'
-
!
(2) On February 12 and 14, 1991, the inspector requested that eight
!.-
torque wrenches available for issue in the PM calibrated tool
!
issue room be checked at one point of their, useable range for
~
calibration.
Licensee personnel performed a' check on each of i
these wrenches. Two of the eight (255).i 393 and f381. were
!
found out of calibration. Torque wrench f3g3 indicated 240 j
j in-lbs at an actual value of 218 in-1bs, while torque wrench'
- 381 indicated 125 in-lbs at an actual value of 145 in-lbs.
'
i Both wrenches were indicated by their calibration records to i
be in calibration. The tolerance for torque wrenches to be i
considered in calibration was four percent of their setacint j
per Step 7.2.7 of procedure MP M-53.1.
!
Licensee personnel could not explain this finding. They
!
indicated that typically one out of 100 times they would
-
find a similar discrepancy. The discrepant. torque. wrenches
.
were removed from service for calibration.
,
i
?
The inspector noted that Maintenance Procedure M-53.1 i
Revision 5. Section 7.4 required that torque wrenches be i
verified before and after their use at the setting, or over j
the range, at which they were to be used..The inspector noted that it was possible that a' torque wrench could be in calibration j
at some points and out of calibration at others.
In this case.
j a simple one point calibration check such as that perfonned at i
the inspector's request would not necessarily indicate a j
previous verification error had been made. However, this
!
relatively high percenta9e of discrepancies suggested
a significant percentage of verification errors.
" The inspector concluded that the verification program for Pfl torque wrenches was potentiall i
!
calibrated tools were in use. y failing to ensure that only'
(50-275/91-04-03).
This is an open item i
<
(3) The inspector observed the DM M&TE personnel perfom.
,
.
j type)typetorquewrenches,alsoknownasclicktype"seve
{
)
j torque wrenches.
71.ue verifications were perfomed on t
i different models of the Williams Torque Wrench Tester.
7.2.2 and 7.3.1 of procedure Mp M-53.1 referenced in theSteps'
!
i previous p6ragraph, required the calibration of torque These personnel were not aware of the prominent N
!
Technical Bulletin No. T8-129. the applicable Technical
!
Bulletin for this activity, which described the proper
_
use of this equipment:
" Extreme care must be exercised when checking audible -
.
'
indicating (snap-type) wrenches so the o
-
not pull beyond the " break away torque."perator does
.
_...
. _.
.
.
-.
. -. -
.
.
.. -..
.
.
--
..
. - -
-
2-
,.
l-
)
.
The inspector observed that m'M&TE personne'l were unaware i
of this precaution, and routinely did not'use extreme care, Consequently, for smaller torque. wrenches. measuring-in j
I in-lbs..these t.rsonnel routinely pulled the' wrench
'
!
significantly beyond the break away torque..For example, for a wrench set to break away at 100 in-Ibs, the Williams
,
j Tester typically' indicated a peak torque value of-130-140-
in-Ibs had been attained during the test. For. larger torque:
wrenches, the same effect was not noted. For example. for
a wrench set to break away at 100 ft-Ibs, a peak torque value
,
j of 102 ft-Ibs was typically attained. The person performing
the test attempted to estimate the value of. the release torque.
i but the process was inherently imprecise given the relative-rapidity with which the wrench passed through the break'away torque value.
i The inspector noted that field personnel could reasonably be expected to be even less ' meticulous than the testing i
j personnel who routinely exceeded the~ setpoint torque values i
unintentionally. The inspector concluded that this testing had demonstrated that use of snap-type torque wrenches in i
the in-lb ranges could result in significant overtorquing of i
,
fasteners in those ranges.
A related. but broader-i conclusion was independently reached by the licensee's
Technical and Ecological' Services (TES) Division Report
4200C9176. (January 14.1991) issued in response to_ Action
>
l RequestA0183483(March 17,1990). That' report made one observation which was relevant to this discussion:
" Click" type wrenches may have a second [ tor
that can significantly over torque a bolt." que] peak
t j
The inspector determined that the ISC M&TE program had i
recently greatly reduced the number of snap-type Mue i
i j
wrenches available for use, whereas the pH MATE program had not addressed the concern.' To the contrary, on February 14
1991, the inspector was advised by Itcensee testing personnel
-
i that workman were requesting snap-type torque wrenches.for
'
i use inside the RCA from the PM MATE calibrated tool 1s =
room, despite'the licensee's decision discussed in Section 3.A
{
-
above to use only ISC calibrated tools in the RCA beginning i
January 14, 1991.
-
,
The inspector concluded that, for small torque wrenches, the
licensee's program to control torquing of fasteners had not -
!
been nearly as precise as required by MP M-53.1.
i The
.
!
inspector noted that the effect of the unknown errors
produced should be considered by the licensee for systematic-j i
evaluation. (0penItem 50-275/91-04-01)
i
<
ij I
E i
i j
i
i
,
'
d
.2._a_._._,._.,._._.,_.._..___,.--.._,_,,-,_.-,._,,,,.,..,L
-
--
.-
,-
.
-_-_
[..
o
-
!
-
Review of previous Quality Assurance Audits a.nd Quality control
,
,
- '
5urveillances Related to MM M6TE Program weaknesses (Unresolved Item
.
50-275/90-29-01)
,
,
The inspector reviewed the status of corrective action for the.
i surveillances,~ audits and the inspection which preceded this inspection
-
!
l and which were related to PM M&TE.
Inspection Report Nos. 50-275/90-29-and 50-323/90-29 previously detailed much of this chronology. It is
!
sumartred here, combined with_ additional information. determined during j
this inspection, d
i j
Four licensee reports from December 5.1989 through September 6.1990
~
}
i identified repeatedly that the MM M&TE program had significant weaknesses,
j These reports were:
j Quality Control Surveillance 89-175 dated December 5.1989 a.
j This report concluded that "a significant problem exists'with
!
j
[MM] M&TE traceability to specific work activities." An '
expanded surveillance was recommended, but no definitive
corrective action which addressed correcting this problem j
was indicated by the survelliance.
j b.
Quality Co;. trol Surveillance 90-030 dated April 12. 1990
.4 This surveillance identified a variety of programmatic problems.
These included:
'
!
" chronic omission of data, data errors, missing signatures..
l
.. logs not maintained;
'
)
incomplete and no history searches / evaluation for out-of-
!
tolerance M&TE. history searches' ARs exceeding procedural l
time limits;
)f
.
'
equipment usage not being' consistently recorded, jeopardizing i
the accuracy of the calibration data baseline;.
!
j
"as found" data not recorded and in some tratances photocopied
!
and uses (sic) for multiple tools."
I
.s
'
inadequate training and qualification of calibrators;
.
.
}
extrapolation of calibration data which is not permitted; j
use of standards with less than the [ procedurally] required
{
accuracy;"
-
l l'
i b
i
!
_
-.
__
-
....
~.
-
.
...
. _ _ _
_ ___ __ _
___ _ _. _ _ _
_
_
__
-
.
_
_
_. _
.
u
'q
,
This expanded surveillance of M MATE concluded thatt.
'
-
" Unsatisfactory performance of tool calibration and issue
..
activities was primarily due' to unclear or incomplete..
procedures and poor. practices by maintenance personnel.
Many of the progra* administrator's corrective actions" during the last year only corrected the symptoms of the problems and not their causes thus allowing problems to. recur...
.. The overall effectiveness of the Mechanical Maintenance-calibration program is unsatisfactory."
A follow-up surveillance was recommended by QC to detemine whether the corrective actions identified and scheduled by the m department were effective.
In discussions with the inspector the QC Manager stated that he that a nonconformance report (NCR) be issued. but tha considered that an NCR was inappropriate because the survel'lance thought that some of the surveillance findings were inv At the time of the previous inspection, which began on November 27 of this surveillance had still not been addressed by
,
Maintenance department (e.g. AR A0184108.. dealing with rusted stan and AR A0183542, dealing with widespread recordkeeping errors).- Where the findings had been addressed, they were addressed in piecemea
,
i by lower level personnel, using conuent fields on' Action Requests
.I than in a coordinated way which clearly
, rather
'
oversight and approval of the response. demonstrated management Most significant y, licensee personnel could not provide an overall conclusions and seven'y documentation which addressed the'five
,
During this inspection, this documentation was still not availablere
The inspector concluded that these conclusions and reconnenda i
not been formally addressed.
.
-
i Quality Performance and Assessment Branch Surveillance 90-126 c.
nay 30. 1990 The Site Quality Assurance Manager stated that this. surveillanc i
initiated by Quality Assurance because of concerns regarding the
'
validity of QC Surveillance 90-30 This surveillance was a r.
Quality Evaluations-(QEs) related to the program f eview of
.
,and
surveillance clearly identified that some of the findings identified
'
This by QC were repetitive, and recommended a QA-audit of the area
,
This surveillance mentioned that M M&TE deficiencies
.
identified as early as 1985.
Assurance Audit 85230P, dated OctoberThe inspector confirmed that Quality-failures to properly document Mt.TE usage.17,1985, had identified personnel training. documented to have been corrected by procedural
<
,., -.. -
_
.
-. -
...-
..
-
.
.
ic
'
i,,
Thu inspector observed that surveillance 90-126 did not recorsend j
an NCR for the recurrence of M&TE program weaknesses.
!
>
l
.
-
i j
d.
Quality Assurance Audit 908127 dated seotember 6' 1990
.
This audit independently reassessed the M&TE programs. including
!
m MATE. five months after the PM program deficiencies were identified
by QC. No coment was made regarding the existence of programs with
!
different procedural requirements for use of MATE.' The audit ~did
.
a reconfirm several of the deficiencies identified in the QC
!
Surveillence 90-30. Generally it was less clearly written, and i
made no general conclusions or recomendations regarding the MM M&TE i
j program.
~
i i
The audit also identified recurrent deficiencies related to those j
identified in 1987 by NCR DC0-87-QA-N001. That 1987'NCR referred to i
j the discove;y that the calibration accuracy ratios between calibration
standards and MATE had not. in all cases, been determined and documented.
i l
This 1990 audit found recurrent failures to control. this' ratio. as had j
QC Surveillance 90-30.
I Most of the problems identified by Audit 908127 were limited to~
j technical issues. The most significant findings of this audit
'
i appear to have been:
)
Usage information for MATE was not recorded as required on 351
'
l of 20 work orders reviewed. This was a repeat finding from QC
Surveillance 90-30.
~
l The procedurally required accuracy ratios between calibration
.
standard and M&TE of 4:1 were not attained for several different
types of equipment. This was a repeat-finding from QC
{
Surveillance 90-30.
t
,
j MM did not maintain vendor manuals for most of the equipment
'
they were responsible for calibrating; therefore, tool room personnel could not describe or refer to recognized practices and methods of calibration. This particular j
discrepancy, at sted in paragraph 3.b.3 above, was still evident during this inspection in that tool room personnel
,
were'not familiar with the precaution in the vendor manual for the use of click type torque wrenches.
!
This audit did not state clearly any conclusions regarding the
.
adequacy of the MM MATE program, nor did it make any clear
{
recomendations to management, unlike QC surveillance 90-30.
,
<
!
However, unlike the DC surveillance, which only documented the
specific findings as Action Requests to the 191 department, this.
auditdidissueseveralAuditFindingReports(AFRs),whichrequired i
the MM department to respond with a root cause determination for the l
problems.
i i
i
-
_ _. _ _..., _
_
..
. -...
.. _ _., _..
.., _..,. _
,_,_...__........_......._.__.mc.,-
.
.m....
._.
_ _ _..
__
._ _
-. _. _
__ __. __. _ _.,
_ _ _ _ _
__ _
.
-. _
.
'. l Action Requests were the licensee's working level request for.
information by the requestor of. the affected party, in this case,
the 2 department.
,
At the request of the m department, the due date for these detailed corrective actions was extended. At the time of the previous NRC inspection, no action had been taken on
,
j these findings.
,
breakdown. nor did it refute QC Surveillance 90-30. -
i-
'
action for its findings had been taken prior to the NRC inspection which commenced on November 27, 1990.-
!
5.
Conclusions Regarding Licensee Corrective Action for MM M&TE Program Deficiencies Prior to Insoection 50-F75/90-29 -
T" previous NRC inspection (50-275/90-29) )dentified that a.
nonconformance report-had not been initiated to address the recurrent-I failures to follow the established M MTE program.
That inspection focused on the statements in' Audit 90812T that E had not fully
implemented the corrective actions required by NCR DCO-87-QA-N001.
I As discussed in paragraph 4d above, that NCR's findi
,
j nonconformance was identified in the previous report as unresolved ites
.
90-275/90-29-01.
in addition to those diThe previous inspection also identified several examples, i
j in the t91 M&TE program.scussed in this report, of specific quality problems
After review of the material discussed above, discussion with licensee i
personnel and managers, and the observations discussed in Paragraph 3
the inspector concluded that the licensee's corrective action in respo.
{
to the many formal reports of a deficient M MTE program had been j
nse.
I e.
'
Quality control Surveillance 90-30, in particular. clearly reported a significant condition adverse to quality
.
j failures to implement the m MTE program, procedural requirementsnamely i
licensee's own surveillance report characterized the m MTE progra The j
' unsatisfactory" and "not acceptable."
.
\\
Senior licensee management was provided with the report of sevef 90-30. as was Quality Assurance.
inspection, over seven months laterYet, at the time of the previou.
-
i issued for these programmatic discre,pancies.a nonconformance report had not been As a direct result, the c.ause for the weaknesses had not been dettraineti 'an to restore program quality were not defined.
and an additional surveillance were performed which added addition These subsequent efforts reduced the clarity of i
j conclusions and recoment!ations, and postponed and diluted effect i
_
.
corrective action to correct the overall problem.
.
.
.
,
g9,.as
- 4y=.
59"C#
. _. -
-
-.
.-
---
.
--.
--
-
..
..
.-
-
.-
._
'
'
- The inspecter, therefsre, concluded that the scope and breadth of these failures was substantial enough to indicate a breakdown in the Quality
!
Assurance program for the calibration and control of measuring and test y,
equipment. This is an apparent violation of the requirements af 10 CFR part 50. Appendix 8. Criterion XVI. " Corrective Action, and Quality
!
Assur., ace Procedure 15.8. "Nonconfomances." Paragraph 2.1.1 and 3.1.- in that a nonconformance report for this breakdown was not initiated until after the previous NRC inspection again brought this problem to senior
.;
management's attention (Enforcement Item 50-275/91-04-02).
-
Unresolved Item 50-275/90-29-01 which directly referenced the calibration ratto discrepancies NCR is correspondingly incorporated as part of this apparent violation, and is closed.
As noted above, related issues of potential overtorquing of-small fasteners due to use of click-type torque wrenches'(0 pen Item 50-275/91-04-01), use of uncalibrated torque wrenches (0 pen Item 50-275/91-04-03), and undocumented use of calibrated tools (0penItem 50-275/91-04-04) were identified. These items are considered integral parts of the Nt MATE program breakdown..They will be followed up during followup on the Enforcement Item.
6.
Review of Licensee Corrective Action Since the previous Inspection As imediata corrective action for the findings of Inspection 90-29/90-29 toe licensee issued nonconformance report NCR 0C0-90-MM-N089 on December 21, 1990. The nonconformance was described as:
" Previous QC and QA audit findings indicated a significant number of program implementation deficiencies. These deficiencies, and the time involved responding to the deficiencies warrant
,
further evaluation."
At the conclusion of the latest inspection, this NCR was still under revision to determine the appropriate corrective actions. Most of the
.
proposed corrective actions from the most recent QA au: fit. 90812T. had been delayed pending the development of a unified corrective action plan from this NCR.
,
7.
Unresolved item
'
An unresolved item is a matter about which more information is required to ascertain whether it is an accept.ble ites, a deviation, or a violation.
8.
Exit Interview The inspector met with licensee management denoted in Paragraph 1 on-February 14, 1991.. The scope and detailed findings of the. inspection were discussed.
Licensee representatives acknowledged the findings of the inspection.
Subsequently.'on February 25. 1991 licensee representatives were informed that an Enforcement Conference on,the results of this inspection would be conducted on March 8.1991, i
l.
,.
.-
-.
.
- -
.
...
-
- -.
-.
.