GENERAL INFORMATION

MKS Instruments, Inc. INTERNAL AUDIT REPORT

  • GENERAL INFORMATION

  • DOCUMENT TITLE:

  • DATE:

  • TO:

  • CC:

  • FROM:

  • SUBJECT:

  • SITE:

  • LOCATION:
  • AREA TO AUDIT:

  • AUDIT/DOCUMENT REPORT #:

  • INVOLVED PERSONNEL:

  • DOCUMENT PREPARED BY:

AUDITEES

Personnel that was audited during the process

  • Auditees:

  • Person
  • Add signature

PREVIOUS FINDINGS

Review of Previous Findings

  • Are there Non-Conformances/Findings to review from previous audits (CAR)?

  • Non-Conformances/Findings Actions from previous audits (CAR):

  • Finding
  • CAR Approved and Closed?

  • What is the status of the CAR?

  • CAR assignee:

  • Need to escalate?

  • Escalate to whom?

  • Are there observations from previous audits that need review?

  • Observation from previous audit

  • Observation
  • Was the observation still present during the assessment?

  • Need to assign observation as a Non-Conformance/Finding?

  • Non-Conformance

  • NC

AUDIT SECTIONS

  • SECTION TO AUDIT:

4 QUALITY MANAGEMENT SYSTEM

4.1 General & Documentation Requirements

  • This sections assesses the organization's ability to document the quality management system.

  • 1. What is your current ISO certification status?

  • Certification expiry date:

  • 1.1 Identify standards certified to:

  • ISO 9001:2008

  • AS9100

  • TS16949

  • ISO13485

  • Other.

  • 1.2 Identify scope of certification(s) (i.e. Any exclusions).

  • Certification expiry date:

  • Root Cause for being Past Due?

  • Does the assessed have a estimated date for Re-Certification?

  • Scheduled Re-certification date:

  • Need to escalate?

  • To whom?

  • 1.1 Identify standards certified to:

  • ISO 9001:2008

  • AS9100

  • TS16949

  • ISO13485

  • Other.

  • 1.2 Identify scope of certification(s) (i.e. Any exclusions).

  • 2. How are outsourced functions that affect product or service conformity identified and implemented with effective controls?

  • Describe actions selected above.

  • 3. What is your process for your document control system (ISO 4.2.3, a-g).

  • 3.1 Document control system includes:

  • - Procedure for approval.

  • Description of process:

  • - Change or Revision control.

  • Description of process:

  • - Identification & legibility.

  • Description of process:

  • - Control for externally originated documents (i.e. Customer documents, specifications, industry standards, etc.).

  • Description of process:

  • - Obsoleting documents.

  • Description of process:

  • - Obsoleting documents.

  • Description of process:

  • - Ensure access to controlled documents when/where needed.

  • Description of process:

  • - Process is in place to periodically verify that the latest customer revisions are being used (Drawings & Specifications).

  • Description of process:

  • 4. How are your quality management system procedures defined, implemented and maintained? (ISO 4.2.1.b,c,d) (ISO 4.2.2.b).

  • 4.1 ISO requires at a minimum documented and implemented procedures for the Control of Nonconforming Product, Corrective Action, and Preventive Action (do not have to be individual procedures - they can be combined).

  • 4.2 Level 2 procedures should be referenced in the form of an index.

  • 4.3 Clear ownership of documents with process defined for regular review (look for evidence of revision changes).

  • 4.4 Changes to the quality management system are effectively planned and communicated (to include training as applicable).

  • 4.5 Process implemented for periodic review and update of documents per a defined frequency.

  • 5. How do you demostrate integration/interaction of the Quality Management System business process? (ISO 4.2.2.c).

  • 5.1 Quality manual has a disgram showing how all processes in the Quality Management System interact.

  • 5.2 May also be shown on process maps such as SIPOCS (Supplier Input Process Outputs Customer).

  • 5.3 Flowcharts on process documents may also show interaction with other processes.

  • 6. How do you ensure quality records are identified and effectively managed? (ISO 4.2.1; 4.2.4).

  • 6.1 Control of Records procedure is required that addresses identification of records, storage, protection, retrieval, retention and disposition.

  • 6.2 Records requirements are clearly defined to include duration and location of storage (i.e. record retention matrix).

  • 6.3 Records should demostrate operational effectiveness and conformity to requirements.

  • 6.4 Sample several records to ensure that the process is being followed (this can often be sampled as other sections are being assessed.

  • 6.5 Verify that records are legible, readily identifiable, and retrievable.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

5 MANAGEMENT RESPONSIBILITY

  • SUB-SECTION:

5.2 Customer Focus

  • This section assesses the organization's ability to focus on meeting customer requirements.

  • 1. How do you determine your customers needs and requirements, monitor performance, and drive subsequent improvements? ( ISO 5.2, 8.2.1, 8.4)

  • 1.1 Organization clearly defines who their internal and external customers are

  • 1.2 Customer performance requirements and goals are established.

  • ( Identify existing metrics on the "Metrics" worksheet - at minimum must include quality and delivery performance, must include AMAT supplier balanced scorecard when applicable)

  • 1.3 Data is analyzed to determine top issues inhibiting the consistent achievement of defined goals.

  • 1.4 Improvement actions are defined and implemented when goals are not being met.

  • 1.5 Performance trends are identified, trended, analyzed, and improved towards goals as appropriate.

  • 2. How do you assess and improve your customers satisfaction levels?

  • 2.1 Process defined for conducting customer satisfaction surveys on periodic basis (minimun yearly).

  • 2.2 Survey results are analyzed and corresponding improvement plans are established towards achieving defined goals.

  • 2.3 Improvement plan deployment is monitored.

  • 2.4 Closed loop communication process with Customer on improvement plans and progress.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

5.3 Quality Policy

  • This section assesses the organization's quality policy.

  • 1. How is your policy defined, approved and implemented? ( ISO 5.3)

  • 1.1 Quality Policy defined and approved by top management.

  • 1.2 Includes a commitment to "comply with requirements and continually improve the effectiveness of the quality management system".

  • 1.3 Quality Policy is effectively communicated throughout the organization ( i.e. as part of new hire orientation, postings, badges, screen savers, etc.)

  • 1.4 Personnel are aware of the quality policy, know how to access it, understand how it applies to their jobs.

  • 1.5 Quality Policy is reviewed to ensure it continues to be suitable and appropriate for the organization.

  • 1.6 Quality Policy provides a framework for the Quality Objectives (i.e. linkage between policy and how the quality objectives measure the deployment of the policy)

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

5.4 Planning

  • This section assesses the organization's ability to establish strategic quality planning.

  • 1. What is your process for the establishment and deployment of Quality Objectives. ( ISO 5.4.1)

  • 1.1 Management reviews and defines Quality Objectives annually at a minimum.

  • 1.2 Quality Objectives address at minimum supplier performance, internal quality performance and performance to customers.

  • 1.3 Quality Objectives are translated into measurable metrics with goals established.

  • ( Identify Quality objective metrics on the "Metrics" worksheet)

  • 1.4 Goals are sufficientaly challenging and modified as required to drive continual improvement.

  • 1.5 Quality Objectives are cascaded into " relevant" and "measurable" functional/department and individual employee objectives.

  • 2. How do you provide for the monitoring, analysis and improvement of Quality Objectives?

  • 2.1 Management regularly reviews progress per as defined frequency that is effective ( i.e. yearly is insufficient if goals are not being met).

  • 2.2 Metrics are trended for performance monitoring towards goals.

  • 2.3 Data is analyzed to determine top issues inhibiting the consistent achievement of defined goals.

  • 2.4 Improvement actions are defined and implemented when goals are not being met.

  • 2.5 Effective visual communications of Quality objectives and status throughout the organization ( i.e. top level status boards, department level status boards).

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

5.5 Responsibility, Authority and Comunication

  • This section assesses the organization's ability to ensure that the responsabilities and authorities are defined and communicated within the organization.

  • 1. How do you define and communicate responsabilities, authorities and their interrelation within the organization (ISO 5.5.1)

  • 1.1 Top management ensures that the responsabilities and authorities are defined and communicated throughout the organization ( defined roles and responsabilities, organizational charts, responsability sections in procedures, etc.)

  • 1.2 Personnel in the organization are aware of roles and responsabilities.

  • 2. Do you designate a Quality Management System representative with defined responsabilities. ( ISO 5.5.2-3)

  • 2.1 Quality Management System management representative has been appointed who, irrespective of other responsabilities shall have responsability and authority for the QMS.

  • 2.2 Representative is part of the management team.

  • 2.3 Representative reports to top management on the performance of the Quality Management System.

  • 2.4 Representative promotes awareness of customer requirements throughout the organization.

  • 2.5 Personnel throughout the organization are aware of who the QMS management representative.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

5.6 Management Review

  • This section assesses the organization's ability to review the organization's quality management system.

  • 1. How do you ensure that the Quality Management System unerdgoes periodic management review to review progress and drive continual improvements? (ISO 5.6)

  • 1.1 Management reviews are performed at a frequency necessary to drive continual improvements ( i.e. some topics may be reviewed yearly ( i.e. quality policy suitability) but some topics may require more frequent view ( i.e. audits, corrective actions, customer sat, performance metrics, open action items, etc.)

  • 1.2 Records of management reviews are maintained per records retention requirements ( i.e. agenda, meeting minutes, action items).

  • 1.3 Effective process in place for action item follow-up to ensure closure in a timely manner.

  • 2. What are the key topics covered during the management review? At minimum, they should include the following; ( ISO 5.6)

  • 2.1 Planned changes to the Quality Management System.

  • 2.2 Quality Policy review.

  • 2.3 Quality objectives review.

  • 2.4 Results of audits ( internal and external).

  • 2.5 Customer feedback ( performance metrics and satisfaction surveys).

  • 2.6 Process performance ( i.e. key manufacturing yields, key manufacturing process characteristics (Cpk's), metrics of key bussiness process, etc.).

  • 2.7 Product conformity ( i.e. supplier non-conformances, FAI yields, in-process inspection, final inspection, 1st piece checks, non-conforming materials data, scrap, Cost of Quality, etc.).

  • 2.8 Status of preventive and corrective actions.

  • 2.9 Follow-up action status from previous reviews.

  • 2.10 Recommended improvements to the Quality Management System.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

6 RESOURCE MANAGEMENT

  • SUB-SECTION:

6.2 - Human Resources

  • This section assesses the organization's ability to improve itself using human resources.

  • 1. How do you ensure that a training program is defined, implemented, monitored, effective, and continually improved? ( ISO 6.2)

  • 1.1 Documented training requirements forvall job families that include general courses/skills and job family specific training requirements ( i.e. training matrixes, job family training curriculums).

  • 1.2 Progress towards meeting training requirements is planned and monitored ( i.e. reports that show what has been completed and what is still needed).

  • 1.3 Training requirements are used for employee development planning ( i.e. Training plans exists for identified needs, Employee goals and objectives address needed training).

  • 1.4 Managers/ Supervisors utilize training completion status to determine which personnel are certified and qualified to perform given tasks and responsabilities.

  • 1.5 Training records are maintained.

  • 1.6 Training metrics are identified, trended, analyzed, and improved towards goals as appropriate.

  • ( Identify existing metrics on the "Metrics" worksheet - at minimum, there must be metrics or reports that show the organizations overall training completion rate for "all" training requirements, " continuous improvement methods/ tools" training, and " AMAT required" training).

  • 2. What is your process for on the job certification for operators?

  • 2.1 Documented process for on the job certification of manufacturing personnel ( not limited to specialized skills such as welding, soldering, assembly work).

  • 2.2 Contains criteria for certification and defines training methodology ( i.e. qualified trainers assigned, observe work, be observed, approvals before independent work performed).

  • 2.3 Considers Mastery levels where appropriate ( i.e. Certified to perform work, certified to teach others).

  • 2.4 Contains requirements for periodic recertification and decertification when needed ( i.e. Job change, performance issues, away from job for a certain time period, etc.)

  • 2.5 Certification status and plans are monitored by management ( i.e. Training matrix showing both employee certification status and plans to address gaps and employee development)

  • 3. What is your process for Continual Improvement tools training?

  • 3.1 Continual Improvement methods and improving tools course curriculum has been developed based on organizational needs analysis and strategic planning ( i.e. Lean / Six Sigma methodology, kaizen methodology, 8D methodology and all corresponding improvement tools needed to support improvement methodologies).

  • 3.2 Curriculum has been integrated into all job family training requirements as applicable to each job family.

  • 3.3 Curriculum identifies both the skills and the corresponding training courses ( can be internal and external courses).

  • 3.4 Training materials are available in support of training courses.

  • 3.5 Training completion metrics or reports are monitored by management.

  • 4. What is your process to ensure relevant personnel undergo AMAT specific training?

  • 4.1 Inventory of applicable customer requirements has been performed and requirements have been defined ( i.e. Copy Exact, workmanship standards, cosmetics, packaging/crating, product safety, 8D, FAI, PQP, CofC, FIR, DDD, SSQA, specifications listed on part drawings, etc.).

  • 4.2 Training requirements have been integrated into the applicable job family training requirements/ curriculums.

  • 4.3 Evidence that personnel working on AMAT orders have all completed required training before operating independently.

  • 4.4 Refresher training requirements are defined and implemented ( i.e. Yearly CE training).

  • 4.5 Training completion metrics or reports are monitored by management.

  • 5. How do you ensure that training is effective?

  • 5.1 Methods are defined to evaluate the effectiveness of training ( i.e. testing, internal/ external competency based certifications, observation & feedback, performance evaluations, defect mapping, overall training needs analysis, etc.).

  • 5.2 Effectiveness of training is evaluated per the defined method.

  • 5.3 Improvement actions defined and implemented forbtraining effectiveness gaps.

  • 5.4 Management periodically reviews overall training program effectiveness and drives continual improvements to training program content and/ or methods for ensuring effectiveness of training ( i.e. Defect rates related to workmanship errors, processes not being followed, problem solving with inadequate root cause analysis, etc.).

  • 6. How are organizational goals and objectives established and cascaded throughout the organization?

  • 6.1 Management defines Goals and Objectives on a yearly basis.

  • 6.2 Quality Objectives are included in and/ or aligned with Goals and Objectives.

  • 6.3 Goals and Objectives are translated into measurable metric with goals established.

  • ( Identify top level goals and objectives metrics on the " Metrics" worksheet).

  • 6.4 Goals are sufficiently challenging and modified as required to drive continual improvement.

  • 6.5 Goals and Objectives are cascaded into "relevant" and "measurable" functional/department and individual employee objectives.

  • 7. How are your goals and objectives monitored, analyzed, and improved?

  • 7.1 Management regularly reviews progress per a defined frequency that is effective.

  • 7.2 Metrics are trended for performance monitoring towards goals.

  • 7.3 Data is analyzed to determine top issues inhibiting the consistent achievement of defined goals.

  • 7.4 Improvement actions are defined and implemented when goals are not being met.

  • 7.5 Effective visual communications of Goals and Objectives and status throughout the organization ( i.e. Top level status boards, department level status boards).

  • 8. How do you determine and improve employee satisfaction?

  • 8.1 Process are in place for employee satisfaction surveys ( results analyzed, improved plans defined and implemented, and closed loop results monitored for employee satisfaction improvements).

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

6.3 - Infrastructure

  • This section assesses the organization's ability to determine, provide and maintain appropriate infrastructure.

  • 1. What is your process to determine, periodically review and fund the required infraestructure requirements to meet product/ process requirements? ( ISO 6.3) Includes;

  • 1.1 Buildings, workspace and associated utilities.

  • 1.2 Layout of equipment to improve workflow efficiency.

  • 1.3 Process equipment ( hardware and software). This requirement is also included in the new product introduction process.

  • 1.4 Supporting services ( e.g. Transportation, communication, IT, etc.).

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

6.4 - Work Environment

  • This section assesses the organization's ability to determine and manage the work environment.

  • 1. How do you determine and periodically review the work environment needed to meet product/ process requirements? ( ISO 6.4)

  • 1.1 Safety and ergonomic requirements with the appropriate personal protective equipment. Audits are performed to check compliance.

  • 1.2 Environmental controls are in place and monitored to meet production requirements ( e.g. Temperature, humidity, etc.).

  • 2. If applicable, what ESD process controls been identified and implemented?

  • 2.1 ESD controls, equipment, and training.

  • 2.2 Audits are performed to ensure ESD compliance. Records are maintained.

  • 3. If applicable, what cleanroom controls been identified and implemented?

  • 3.1 Protocol requirements for people, equipment and material entering the cleanroom.

  • 3.2 Effective particle monitoring program for all areas of the cleanroom.

  • 3.3 Cleaning requirements of the cleanroom.

  • 3.4 Periodic compliance audits.

  • 4. How do you utilize 5S to continually improve the work environment?

  • 4.1 Documented process for 5S and deployed throughout the organization.

  • 4.2 Top management communicates their support of 5S to the organization.

  • 4.3 Required 5S training is identified and given to all applicable employees and included in their training records.

  • 4.4 5S area audits with goals are performed on a periodic basis and results are posted. Action items are taken if goals are not being met.

  • 4.5 Rewards and recognition are tied to 5S improvement.

  • 4.6 Evidence of logical and effective visual management ( i.e. defined locations for materials and equipment ( staging areas, kanbans), signage, tool shadow boards, organized work areas, general communication boards, work areas metrics/ action boards, etc.).

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

7 PRODUCT REALIZATION

  • SUB-SECTION:

7.2 - Customer Related Process

  • This section assesses the organization's ability to determine and review customer requirements and communication.

  • 1. How does your contract review process meet the following requirements? ( ISO 7.2)

  • 1.1 Defines customer requirements, including delivery and post-delivery activities ( e.g. Warranty, maintenance services, recycling, final disposal, etc.).

  • 1.2 Defines non-specified requirements but necessary to satisfy the customer ( e.g. Reliability tests, functional tests, etc.).

  • 1.2 Defines statutory and regulatory requirements.

  • 1.3 Defines organization requirements ( e.g. Skillsets, equipment, etc.)

  • 1.4 Records of the contract review including changes are maintained.

  • 1.5 Contract review process metrics are identified, trended, analyzed, and improved towards goals as appropriate ( i.e. quote cycle times, quote win rates, customer feedback of contract review process, corrective SPS's etc.)

  • NOTE: Identify existing metrics in the "Metrics" Section.

  • Auditing Repair Suppliers?

  • Controls to ensure that supplier is servicing to the latest revision levels of the OEM specifications).

  • 2. How are customers requirements reviewed for concurrance before commiting to supply product? ( ISO 7.2.2)

  • 2.1 Appropriate functions review and approve the defined requirements to ensure they can be met.

  • 2.2 Any requirements that are not clear or cannot be met are documented and resolved with the customer.

  • 2.3 Review process is supported by a checklist ( or equivalent) that clearly documents requirements, review and acceptance of requirements with appropriate stakeholders, and closed loop resolution of any gaps, questions, or discrepancies.

  • 3. When the contract is changed how does the organization identify what documents are affected and make the required changes? ( ISO 7.2.2)

  • 3.1 Users are identified and made aware of the changes.

  • 4. How does the organization's customer contract (s) establish two-way communication with the customer? Communication topics include: (ISO 7.2.3)

  • Product Information

  • Describe process.

  • Enquiries, contracts including changes

  • Describe

  • Customer feedback, including customer complaints.

  • Describe

  • 5. How do you meet the requirements of the Copy Exact Management process for AMAT?

  • 5.1 Complete AMAT on-line CE training completion ( minimum one person from management, engineering, purchasing, manufacturing, within last year)

  • 5.2 All quality related personnel should complete AMAT on-line Copy Exact training.

  • 5.3 CE refresher training requirements defined and implemented internally and with the supply base ( evidence of implementation on a minimum yearly basis)

  • 5.4 Copy Exact controls are integrated into Internal change management processes and are systemic to the extent possible.

  • 5.5 CE training deployed to the supply base.

  • 5.6 A review of process of adherence gaps between sub-tier and AMAT requirements has been completed and actions established to bring requirements in line.

  • 6. How do you meet the requirements of the AMAT SPS and SNF change communication processes?

  • 6.1 Appropriate personnel are trained in the use of the SPS and SNF communication tools via GMOX. Verify understanding via a log-in.

  • 6.2 Relevant personnel demonstrate understanding of the change requirements matrix.

  • 6.3 Evidence that 180 day advance notification to Applied Materials is being given for all product changes affecting design/ material/processing/mfg source/build locations changes.

  • 6.4 SPS's and SNF's are proactively monitored and managed for effectiveness ( i.e. Monir quantities, reasons for submission, cycle times to close, aging of open requests, open actions items, etc.)

  • 6.5 Effective Change Communication methods have been deployed to suppliers ( i.e. Frmal communication of changes, formal reviews/ approvals, spreadsheet/ database tracking of submissions, etc.)

  • 6.6 Need for SPS's are identified proactively vs. reactively ( i.e. During contract review versus after already in production)

  • 7. What is your process to ensure the requirements of the AMAT First Article Process are met?

  • 7.1 Appropriate personnel are trained in FAI process requirements and usage of the FAI ACA tool on GMOX

  • 7.2 FAI acceptance rates are trended, data analyzed when goals are met, and improvement plans implemented.

  • 7.3 CCR's are submitted for BOM items on assemblies with internal FAI's maintained for the components.

  • Auditing Repair Suppliers?

  • Ensure supplier is in compliance with FRI requirements ( First Repair Inspection).

  • Is section 5.2 part of the assessment scope?

  • 8. How do you determine your customers' need and requirements, monitor performance, and drive subsequent improvements?

  • 8.1 Customer performance requirements and goals are established.

  • ( Identify existing metrics on the "Metrics" worksheet - at a minimum must include quality and delivery performance, must include AMAT supplier balanced scorecard when applicable).

  • 8.2 Data is analyzed to determine top issues inhibiting the consistent achievement of defined goals.

  • 8.3 Improvement actions are defined and implemented when goals are not being met.

  • 8.4 Performance trends are identified, trended, analyzed, and improved toward goals when applicable.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

7.3a - Design and Development (OEM Suppliers Only)

  • This section assesses the organization's ability to control the design and development of product.

  • 1. How does your documented design and development process adress the following requirements? ( ISO 7.3)

  • 1.1 Defines key tasks, responsabilities, and deliverables.

  • 1.2 Tasks and deliverables should align with the requirements of all the questions below.

  • 1.3 Contains defined stages ( e.g. alpha, beta, gamma, etc.) with design review, verification, validation, approvals, for each phase/ stage against defined exit criteria.

  • 1.4 Contains formal sign-off at gates with exceptions and risks documented.

  • 1.5 Senior Management provides design & development project oversight, utilizes cross functional teams, and customer involvement ( where applicable).

  • 1.6 Project Management techniques are in place to track deployment ( actions, owners, dates, status, metrics, etc.).

  • 1.7 Design and development standards and rules are for hardware, electrical, control systems, etc. are documented, controlled and used for all designs.

  • 1.8 Design factors include affordability, quality, reliability, EHS, maintainability, modularity, scalability, ease of troubleshooting, engineering revision, manufacturability, and design risk.

  • 1.9 Knowledge, use and alignment with international and industry standards ( e.g. SEMI S2, S8, S21, S23, SMEMA, etc.)

  • 1.10 Records retention requirements are defined.

  • 2. What metrics are in place to measure, monitor, analyze, and improve the Design and Development process? Examples of metrics:

  • Effectiveness ( post release audit score, lagging quality, controlled build results, yields at approval stages, yields at key review and validation steps, corrective ECO's post release, actual vs. predictable reliability, warranty data, anual failure rates, internal/ external customer feedback).

  • Describe metrics selected above

  • NOTE: Identify existing metrics on the "Metrics" section.

  • Efficiency ( performance to development schedule, performance to cost, development cycle times).

  • Describe metrics selected above.

  • NOTE: Identify existing metrics on the "Metrics" section.

  • 3. How are inputs for product requirements identified, to include the following? ( ISO 7.3.2)

  • 3.1 Product function or performance requirements.

  • 3.2 Statutory and regulatory requirements.

  • 3.3 Previous/ similar designs ( where applicable).

  • 3.4 Other customer/ market specific requirements. Examples; Market requirements specification, Product performance specification, Customer/ statutory/ regulatory requirements, post mortem/ lessons learned reports from previous designs, etc.

  • 3.5 Sample for evidence of records that supports inputs for product requirements.

  • 4. How are outputs of product requirements identified, to include the following? (ISO 7.3.3)

  • 4.1 Verification to design and development inputs.

  • 4.2 Appropriate information for purchasing, production, and service provision.

  • 4.3 Product acceptance criteria ( which includes functional testing).

  • 4.4 Characteristics of the product that are essential for its safe and proper use. Examples; Design specification, Product specification sheet/ catalog, Test Specifications, Sourcing plan, BOM, Safety requirements, environmental requirements, etc.

  • 4.5 Sample of evidence of records that support outputs for product requirements.

  • 5. How is design verification performed in accordance with planned arrangements to ensure that the design outputs have met the design and development input requirements? ( ISO 7.3.5)

  • 5.1 Planned verification is conducted , records kept, and reviewed and approved by appropriate parties.

  • 5.2 Deficiencies found during the verification are documented, reviewed and dispositioned accordingly.

  • NOTE: Verification is the process of determining whether the system is in compliance with its specifications ( design review, performance in simulated real world conditions, marathon testing, halt testing, etc.)

  • 6. How is the design validation performed to confirm that the product is capable of meeting the requirements for the specified application or intended use? ( ISO 7.3.6)

  • 6.1 Planned validation is conducted, records kept, and reviewed and approved by appropriate parties.

  • 6.2 Deficiencies found during the validation are documented, reviewed, and dispositioned accordingly

  • NOTE:
    Validation is the process of determining if the system works in real world applications ( customer confirmation in real world applications, beta test results (prototype at customer site), etc.)

  • 7. How do you ensure the identification and qualification of necessary tools, production & test equipment needed for manufacturing?

  • 7.1 Qualification plans are de<br>Fined and implemented for tools and equipment identified ( i.e. Mchine capabilities studies, test fixture qualifications, gage R & R, etc.)

  • 7.2 Tools and equipment are qualified before use for volume production.

  • 8. How do you ensurebthat product inspection and/ or test criteria is defined and implemented ( receiving, in-process and final)?

  • 9. How do you identify critical to quality characteristics and ensure that the process capability ( Cp, Cpk) is assessed?

  • 9.1 Evidence that process capabilities are determined, reviewed and appropriate actions are taken ( i.e. Risk mitigation actions and improvement projects for process capabilities < 1.33 before volume manufacturing release.

  • 10. How do you ensure operational routers ( traveler, job sequence) are developed and that the content is adequate to assure quality of product before moving to the next operation?

  • 11. How does the design and development process ensure that FMECA ( Design and Process) is conducted?

  • 11.1 Sould not be only for AMAT defined critical PQP parts.

  • 11.2 Cross-functional approach should be used.

  • 11.3 Personnel should be trained.

  • 11.4 Evidence that corrective actions are taken when RPN ratings do not meet defined requirements.

  • 11.5 Poka Yoke ( error proofing) is considered for the corrective actions to as required by high RPN's on FMEA's ( i.e. tools, fixtures, jigs, assembly aids, automated controls, software controls, etc.)

  • 11.6 Evidence that the granularity of the process FMEA's are sufficient to identify failure model risks ( i.e. Conducted a task level not just at overall process level, comprehensive evaluation that looks at potential failure modes related to man, methods, machines, materials, measurement, and mother nature ( environment))

  • 11.7 Documented evidence that process FMEA's are periodically re-evaluated for effectiveness based on on-going data review ( i.e. Process yields, internal defects, external defects, scrap, rework, etc.)

  • 12. How do you ensure that product-specific Process Control Plans are developed?

  • 12.1 Should, not be only for AMAT defined critical PQP parts.

  • 12.2 Process FMEA's are utilized as inputs to process control plan development.

  • 12.3 Evidence that process control plans rely on more robust controls than just inspection ( i.e. error proofing, automation, SPC, etc.)

  • 12.4 Documented evidence that process control plans are periodically re-evaluated for effectiveness based on-going data review ( i.e. Process yields, internal defects, external defects, scrap, rework, etc.)

  • 13. When applicable, how are periodic reliability tests determined and implemented?

  • 13.1 Results are monitored and analyzed, and improvements are implemented as needed ( i.e. weld strenghts, solder joint integrity, peel strenght, coupon testing, destructive tests, functional tests, etc.)

  • 14. How do you ensure that product goes through derating ( electronics), dimensioning, tolerancing and interchangeability assesments?

  • 15. How do you ensure that a trouble shooting guide ( including fault tree analysis) is developed?

  • 15.1 Troubleshooting guide is revision controlled.

  • 15.2 Troubleshooting guide is effectively communicated to users.

  • 15.3 Mechanisms are in place for user support and feedback.

  • 16. How do you ensure that a reliability assesment is conducted ( i.e. Reliability apportionment, MTBF prediction, reliability marathon testing)?

  • 16.1 Actual performance is evaluated against established reliability targets.

  • 16.2 Evidence that improvement actions are taken when targets are not met.

  • 17. How do you ensure that highly accelerated life testing ( HALT) is performed to identify destructive operating limit of the design?

  • 17.1 HALT outputs used to design controls ( i.e. design changes HASS / ESS / burn-in profile to screen infant mortality failures of production parts).

  • 18. How do you ensure transportation shock,mvibration, and drop testing is performed to validate that the packaging is sufficient?

  • 19. How do you ensure a controlled build prior to release for volume manufacturing?

  • 19.1 Controlled build validation requirements are defined and implemented ( i.e. first articles, meeting minimim process capabilities, meeting minimum FMEA RPN requirements, Gage R&R results, etc.)

  • 19.2 Results reviewed and approved prior to volume manufacturing.

  • 19.3 Evidence that controlled build results are used to improve the new product introduction process ( i.e. Integration of lessons learned into process / other products).

  • Product Improvement / Engineering Change

  • 20. How does the change management process address the following requirements? ( ISO 7.3.7)

  • 20.1 Design and development changes are identified and recorded.

  • 20.2 Chenages are reviewed, verified, validated, and communicated internally and externally, as appropriate, and approved prior the implementation.

  • 20.3 Reviews including evaluation of the effects of the changes on constituent parts and delivered products are performed.

  • 20.4 Records are maintained and include results of the reviews and any necessary actions identified as a result of the change review process ( per Record Control process).

  • 20.5 Change management process metric are identified, trended, analyzed, and improved toward goals as appropriate ( i.e. Open/closed, approvals aging, corrective vs. Improvement changes, etc.)

  • NOTE : Identify existing metrics on the "Metrics" worksheet.

  • 21. How does the change management process address the following requirements related to sub-tier suppliers? (ISO 7.3.7)

  • 21.1 The process of how the organization will communicate and control changes of their process to their sub-tier suppliers is documented.

  • 21.2 The process of how a sub-tier supplier will communicate and control changes to their product to the organization is documented.

  • 21.3 These processes are communicated to all sub-tier suppliers and are repeated on a periodic basis. Evidence is available thst sub tiers suppliers comply with stated processes.

  • 21.4 Change Management controls are verified at subtier suppliers for adherence to he organization's stated policies.

  • 21.5 Evidence that sub tier suppliers are notifying the oreganization per established change control processes exits and is traceable.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

7.3b - New Product Introduction (Non OEM Suppliers Only - OEM suppliers score as 'NA')

  • This section assesses the organization's ability to control the design and development of product.

  • DOCUMENTED PROCESS

  • 1. Does the organization have a documented New Product Introduction (NPI) process? (ISO 7.3.1)

  • 1.1 Defined stage with design review, verification and validation.

  • 1.2 Defines key tasks, responsibilities, and deliverables that align with all requirements of this section.

  • 1.3 Defined exit criteria for each phase with formal sign-off gates with documented exceptions and risks.

  • 1.4 Defined record requirements.

  • 1.5 Tasks associated with Customer critical part PQP's are embedded in the new product introduction process.

  • PROJECT MANAGEMENT TECHNIQUES

  • 2. Use of Project Management techniques:

  • 2.1 Senior management provides project oversight and removes roadblocks.

  • 2.2 Detailed project pland developed.

  • 2.3 Project plans updated for progress monitoring.

  • 2.4 On-going evaluation of project risks.

  • 2.5 Maintain action registers and monitor for closure.

  • 2.6 Formal lessons learned analysis.

  • PQP MANAGEMENT

  • 3. If applicable, how do you manage your PQP program to meet the following requirements?

  • 3.1 Critical Parts list is available and shows which parts require PQP's.

  • 3.2 PQP completion status report is maintained and reviewed by management (e.g. inactive part, not started, in process, complete, waiting for Customer approval, Customer approved, etc.).

  • 3.3 Clearly defined action plans (who, what, when) are in place for PQP's not yet completed or approved by Customers.

  • 3.4 Bussiness processes exist with clear roles & responsibilities to meet Customers SPC requirements (i.e. establishing control limits, sampling frequency, measurement system, out-of-control action plans, etc.), projects to work on low CpK areas, metrics to monitor response times and empowered teams to shutdown for frequent out-of-control events.

  • QUALITY PLANNING

  • 4 Are key performance characteristics (KPC) identified?

  • 4.1 Criteria defined on what constitutes a KPC.

  • 4.2 Internal and customer KPC's.

  • 4.3 KPC's are identified in process documentation (e.g. prints, FMEA's, control plans, traveler, router, process documentation, etc.).

  • 5. Is process capability assessed for KPC's?

  • 5.1 CpK goal >1.33 for sample sizes >30?

  • 5.2 CpK requirements for samples sizes are <30?

  • 5.3 Action items defined in process capability is below goal before release to volume manufacturing.

  • 6. Are product inspection, test and acceptance criteria (i.e. receiving, in-process and final test) defined?

  • 7. How do you ensure test specification correlation between the supplier and the Customers Final Test activities is evalated for consistency?

  • 7.1 If any gaps exist, they should be documented with closure actions taken to valume manufacturing.

  • 8. Does the NPI output include the identification and qualification of the necessary tools, production, and test equipment needed for manufacturing?

  • 8.1 Qualification plans are defined and implemented for equipment and tooling (e.g. machine capability studies, test fixture qualification, etc.).

  • 9. Are gage studies conducted on measuring equipment used to measure KPC characteristics?

  • 9.1 Goal >10%. 10%-30% in conditionally acceptable. >30% is not acceptable unless approved by top management.

  • 10. Does the organization have a documented FMEA process?

  • 10.1 Use cross functional teams to create or revise FMEA's.

  • 10.2 Threshold on when corrective actions are required (e.g. RPN>100, criticality>20).

  • 10.3 Use of error proofing techniques (e.g. Poka-Yoke).

  • 10.4 Requirements on when FMEA's need to be revised.

  • 10.5 FMEA training for team members.

  • 11. Are Process FMEA's created or revised during the design process?

  • 11.1 Process FMEA's are used as input for manufacturing documentation (e.g. Control plans, routers, inspection plans, OMS's, etc.).

  • 12. Are product specific control plans created or revised during the design process?

  • 12.1 List the controls used for the manufacturing process control.

  • 12.2 Includes methods for monitoring of controlling KPC's.

  • 12.3 Include SPC control as part of manufacturing steps.

  • 12.4 Develop out-of-control action plans (OCAP) when the process becomes unstable of not statistically capable.

  • 12.5 Control plans are updated when any change accurs that affects product, manufacturing processes, measurement, logistics, supply source or FMEA's.

  • RELIABILITY PLANNING

  • 13. Are reliability controls and improvements implemented as needed (e.g. cable pull tests, weld strengths, solder joint integrity, peel strength, coupon testing, destructive tests, functional tests, burn-in tests, etc.).

  • 14. Is product packaging performed (e.g. transportation shock, vibrations, and drop testing) to validate that the packaging is sufficient?

  • MANUFACTURING DOCUMENTATION

  • 15. Does the NPI output include all required documentation to manufacture the product? (ISO 7.3.3)

  • 15.1 Specifications, drawings, process flow diagram.

  • 15.2 Purchasing information for raw material (e.g. BOM's).

  • 15.3 Manufacturing documentation (e.g. router, set-up instructions, traveler, work instructions, etc).

  • 16. Is all manufacturing documentation validated prior to production release?

  • VALIDATION OF RESULTS

  • 17. Is a controlled build required prior to release for volume manufacturing? (ISO 7.3.6)

  • 17.1 Validation requirements are defined and implemented (e.g. first article inspection, meeting minimum process capabilities, etc.).

  • 17.2 Results are reviewed and approved prior to volume manufacturing.

  • 17.3 Results are used to improve the new product introduction process for other products.

  • 17.4 Records of the validation and any necessary actions are maintained.

  • 17.5 Control Build engineering issues / non-conformances are documented and tracked to full closure.

  • 18. When required by the customer, does the organization have a prototype program?

  • 18.1 Ensures that the same suppliers, tooling and manufacturing processes are used as will be used in production.

  • 18.2 If any processes are outsourced, the organization provides technical leadership.

  • PROCESS, PRODUCT AND PROJECT PERFORMANCE METRICS

  • 19. What metrics are in place to evaluate the effectiveness of the NPI process?

  • 19.1 Validation that product meets customer expected performance requirements/targets (e.g. QPPM, CpK targets, FAI acceptance rates, etc.).

  • 19.2 Validation that product meets internal Quality requirements/targets (i.e Internal defect rates, process capability levels, test yield, controlled build results, etc.).

  • 19.3 Project Efficiency (Cost, Schedule, Cycle Time, etc.).

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

7.4 - Purchasing

  • This section assesses the organization's ability to ensure that purchased product conforms to purchase requirements.

  • Purchasing and Receiving Inspection.

  • 1. How does the documented Purchasing and Receiving processes address the following? (ISO 7.4.1)

  • 1.1 Purchased product must conform to specified purchased requirement.

  • 1.2 Type and extent of supplier control on the purchased product is dependent upon the effect of the purchased product in subsequent steps of the final product.

  • 2. How do you ensure that purchasing information describes the product to be purchased? (ISO 7.4.2)

  • 2.1 Requirements for approval.

  • 2.2 Requirement for qualification of personnel.

  • 2.3 QMS requirements.

  • 2.4 Purchased order is reviewed for adequacy prior to sumission to the supplier.

  • 3. How does receiving ensure that purchased product meets specified purchased requirements? (ISO 7.4.3)

  • 3.1 Defines verification process for incoming material ( e.g. Inspection, FAI, CofC, coupon sampling, independent lab test, etc.).

  • 3.2 Defines how incoming material identified to indicate its status.

  • 3.3 Defines how nonconforming incoming material is segregated from good material.

  • 3.4 Defines how nonconforming incoming product is dispositioned.

  • 3.5 If applicable, defines how statistically supported sampling plan are used for incoming material and is compliant with ANSI/ASQC or comparable method.

  • Supplier Selection & Performance Management.

  • 4. How does the documented supplier Selection and Performance Management process address the following? (ISO 7.4, 8.4)

  • 4.1 Supplier evaluation and selection process is defined. Includes:

  • 4.2 Criteria for selection, evaluation and re-evaluation.

  • 4.3 Quantified criteria for approval to ASL.

  • 4.4 Defines when a supplier should be removed from the ASL.

  • 4.5 Supplier assessment process and plan ( e.g. On-site, survey, etc.).

  • 4.6 Scope includes special process suppliers.

  • 5. How does the organization ensure the following?

  • 5.1 Maintains an ASL that includes the score of the approval.

  • 5.2 Performs supplier assessments according to the defined plan, progress is monitored.

  • 5.3 Reports results of supplier assessments according to the defined plan, progress is monitored.

  • 5.4 Assigns corrective actions when suppliers fail to meet requirements.

  • 5.5 Closes any known performance of capability gaps with suppliers.

  • 5.6 Defines responsabilities for who can approve and disapprove of a supplier.

  • 6. How are the results of supplier evaluations and any necessary actions arising from the evaluation maintained as records ( see Control of Records)?

  • 7. How do you ensure the following special process supplier related requirements are addressed?

  • 7.1 Use of special process suppliers when required by the customer is defined. Includes requirements for those suppliers (e.g. FAI, source inspection, etc.).

  • 7.2 Controls are adequate to ensure that when supplier is performing manufacturing that requires special processes, only Applied Materials approved those suppliers (e.g. FAI , source inspection, etc.).

  • 7.3 Purchasing personnel can demonstrate how to use GMoX to review Applied special processes and approved suppliers.

  • 8. How are supplier performance metrics tracked and used to improve supplier performance in the areas of quality and delivery ( including Special Process)? ( ISO 7.4, 8.4)

  • 8.1 Supplier performance goals arebestablished and communicated to suppliers ( at minimum, quality and delivery requirements)

  • 8.2 Regular performance reviews are held with suppliers warranted ( there should be logical criteria used to determine which suppliers undergo performance reviews - i.e. criticality, spend, performance, high risk, etc.).

  • NOTE: Identify existing metrics on the "Metrics" worksheet.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

7.5 - Production and Service Provision

  • This section assesses the organization's ability to plan and carry out production and service provisions.

  • 1. How do you ensure production documentation and approval requirements are identified? (ISO 7.5).

  • 1.1 All process steps from incoming material through shipping have been identified (e.g. router, shop order, control plan, qualification plan, etc.).

  • 1.2 Required documentation for each step of production is available of point of use (e.g. BOM's, prints, specifications, work instructions, set-up sheets, etc.).

  • 1.3 Process characteristics (including customer specific) have been identified with approval criteria.

  • 1.4 Reaction plans are available for each process characteristic that identifies what to do when it does not pass approval criteria.

  • 1.5 Product release requirements have been identified for each step of production, including delivery and post-delivery.

  • Auditing Repair Suppliers?

  • FOR REPAIR SUPPLIERS:

  • 1.6 Controls are in place to ensure that the supplier is servicing to the latest revision level of the OEM.

  • 1.7 Replacement parts meet or exceed OEM specifications.

  • 2. How do you ensure production support equipment is readily available to support all manufacturing processes (e.g. tooling fixtures, jigs, IMT equipment, etc.)? (ISO 7.5).

  • 2.1 Equipment is verified prior to use (e.g. PM required, calibration, gage R&R, etc.).

  • 2.2 Tooling management program is in place that includes maintenance & repair, tool set-up, equipment manuals, tooling identification that defines status, and required spare parts.

  • 3. What CNC controls re in place?

  • 3.1 CNC Programs are effectively protected from unauthorized changes (through error proofing and not just policy).

  • 3.2 Effective closed loop communication exist on programming problems encountered.

  • 3.3 CNC programming quality is monitored, evaluated and improved as necessary.

  • 4. How do you validate requirements are being met for processes where the output cannot be verified until the product is in use? ( ISO 7.5.2)

  • 4.1 Validate and approve the process to demonstrate the capability of the process to meet product performance criteria ( e.g. Cupon testing, destructive pull testing, functional testing, etc.).

  • 4.2 If any changes are made to any part of a validated process, it is re-validated.

  • 4.3 Records are maintained of process validation.

  • 5. How do you ensure product identification and traceability methods provide traceablity of product back to the following? (ISO 7.5.3)

  • 5.1 Origin of raw materials / parts.

  • 5.2 Equipment / operator performing process steps and key measurements.

  • 5.3 Data collected related to qualitynrequirements ( e.g. Inspection data, test data, etc.)

  • 5.4 Dates processing steps completed.

  • 5.5 Clear and consistent identification of products during product realization ( e.g. Travelers, cards, stickers, etc.)

  • 6. How do you ensure Customer property is controlled for the following? (ISO 7.5.4)

  • 6.1 Process exists to identify, verify, protect and safeguard the property.

  • 6.2 Materials with shelf lives are marked and appropriate controls are implemented.

  • 7. How do you ensure that adequate packaging, handling, and storage of raw and work in process materials is in place to avoid inadvertent damage? (ISO 7.5.5)

  • 7.1 Product or material is preserved during processing and through delivery to prevent it from being damage, lost, unsuitable ( perishable) , or obsolete ( paste shelf life).

  • 7.2 Materials with shelf lives are marked and appropriate controls are implemented.

  • 8. How do you ensure you meet the requirements for the AMAT Decommissioning, Decontamination, and Declaration process ( CORPEHS008)?

  • 8.1 Appropriate personnel are trained on Decommissioning, Decontamination, and Declaration process.

  • 8.2 Controls have been established and implemented to ensure adherence to requirements.

  • 9. How do you ensure you meet the requirements for the AMAT Workmanship Standards (0250-09954)?

  • 9.1 Appropriate personnel are trained on workmanship standards.

  • 9.2 Controls have been defined and implemented for the applicable sections of the standard ( i.e. Employee training, internal workmanship standard documents/ training materials, procedures updated, work instructions updated, etc.)

  • 10. How do you ensure you meet the requirements of the AMAT Corporate Packaging Requirements ( 0250-00098)?

  • 10.1 Appropriate personnel are trained on Corporate Packaging Requirements.

  • 10.2 Controls have been defined and implemented to ensure adherence ( i.e. Employee training, internal packaging instructions, procedures updated, etc.)

  • Is section 6.4 part of the assessment?

  • 11. If applicable, what ESD process controls been identified and implemented?

  • 11.1 ESD controls, equipment, and training.

  • 11.2 Audits are performed to ensure ESD compliance. Records are maintained.

  • 12. If applicable, what cleanroom controls been identified and implemented?

  • 12.1 Protocol requirements for people, equipment and material entering the cleanroom.

  • 12.2 Effective particle monitoring program for all areas of the cleanroom.

  • 12.3 Cleaning requirements for the cleanroom.

  • 12.4 Periodic compliance audits.

  • Critical Part PQP Management

  • 13. If applicable, how do you manage your PQPmprogram to meet the following requirements?

  • 13.1 Critical Parts List is available and shows which parts require PQP's.

  • 13.2 PQP completion status report is maintained and reviewed by management ( i.e. Inactive part, not started, in process, complete waiting for AMAT approval, AMAT approved, etc.).

  • 13.3 Clearly defined action plans ( who, what, whens) are in place for PQP's not yet completed or approved by AMAT.

  • 13.4 PQP's for parts in volume production have been completed and approved by AMAT.

  • 13.5 PQP documentation matches production processes ( compare PQP samples to production routers / travelers)

  • 14. If applicable, how do you ensure effective PQP change management?

  • 14.1 PQP's are periodically evaluated to ensure updates are made and re-qualified when needed ( i.e. FMEA updated periodically with historical quality data, process control plans revised, Gage R&R revisited, site location changes, engineering changes, specification/standards revision changes, etc.)

  • 14.2 PQP's are revision controlled with history of changes maintained.

  • 15. If applicable, how do you monitor, analyze, and improve PQP parts to meet product quality, process quality, or reliability targets?

  • 15.1 Quality/ Reliability performance levels and process capabilities for PQP parts are regularly monitored.

  • 15.2 SPC data is maintained in the AMAT SSPCM tool when required.

  • 15.3 PQP part's in volume production are meeting product quality / reliability and process capability goals.

  • 15.4 Evidence that improvement actions/ projects are identified and progress is monitored when goals are not being met ( i.e. Cpk's < 1.33, QPPM or DPU not at goal).

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

7.6 - Control of Monitoring and Measuring Devices

  • This section assesses the organization's ability to control monitoring and measuring devices.

  • 1. How does the documented Calibration process address the following? ( ISO 7.6)

  • 1.1 Identify the monitoring and measurement that must be taken to provide evidence of product conformity including the IMT equipment needed.

  • 1.2 Establish processesnto ensure that monitoring and measurement can be carried out and is carried out according to defined requirements.

  • 1.3 Requirements clearly defined for qualification and approvals of 3rd party calibration providers.

  • 1.4 Process included for handling tools found damaged and / or accidently damaged during use.

  • 1.5 Formal process is defined for risk and recall analysis on tools/ equipment found out of calibration.

  • 2. How does the calibration program ensure valid results through the following? ( ISO 7.6)

  • 2.1 Calibrate and/ or verify IMT equipment prior use and at specified intervals against traceable standards to NIST or other national standards.

  • 2.2 Document the calibration method used when no traceable standards exist.

  • 2.3 Adjust the IMT equipment to bring it within calibration requirements. Record and maintain before and after calibration measurements.

  • 2.4 Put a calibration sticker on the IMT equipment in a visible location that includes equipment ID, last calibration date, calibration due date and the person that performed the calibration.

  • 2.5 IMT equipment not used to measure product conformity must have a 'reference only' sticker.

  • 2.6 Safeguard the IMT equipment from adjustments that could invalidate the calibration.

  • 2.7 Protect the IMT equipment from damage and deterioration while in use and in storage.

  • 2.8 IMT equipment not in use is tagged and stored to prevent its unintended use.

  • 3. How do you deploy and monitor the calibration program to addressbthe following:

  • 3.1 Maintain a master list of IMT equipment that includes equipment ID, description, calibration interval, last calibration date, calibration due date, location, calibration source and tools status. List includes employee and customer owned tools.

  • 3.2 Status tracking tool is in place and is proactively maintained ( i.e. spreadsheet, database, 3rd party web tool, etc.(.

  • 3.3 On demand reports are available and actively used to manage progress ( i.e. overdue tools/ equipment report, upcoming due tools/ equipment report).

  • 3.4 Calibration process metrics areidentified, trended, analyzed, and improved toward goals as appropriate ( i.e. On-time completion of calibration, % out of tolerance, etc.).

  • NOTE: Identify existing metrics on the "Metrics" section.

  • 4. How is software in IMT equipment verified at defined intervals?

  • 5. How do you maintain records of calibration and verification according to Control of Records requirements? ( ISO 7.6)

  • 5.1 Records are maintained for in-use house and 3rd party calibration.

  • 5.2 Records include equipment ID, NIST standards (or international equivalent), temperature, humidity, technician performing calibration and certification to ISo 17025.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

8 MEASUREMENT, ANALYSIS AND IMPROVEMENT

  • SUB-SECTION:

8.2 - Monitoring and Measurement

  • This section assesses the organization's ability to control monitoring and measurement.

  • Internal Audit Program.

  • 1. How do you ensure the Internal Audit process addresses the following? (ISO 8.2.2)

  • 1.1 A documented procedure is required. Defines the responsabilities and requirements for planning and conducting audits, establishing records and reporting results.

  • 1.2 Defines qualification requirements of the internal auditors.

  • 1.3 Selection of auditors and conduct of audits ensures objectivity and impartiality of the audit process.

  • 1.4 Auditors do not check their own work.

  • 1.5 Auditors are trained and results including in their training records.

  • 2. How do you assure the audit schedule addresses the following? ( ISO 8.2.2)

  • 2.1 Establish an internal audit schedule to determine if the organization's QMS conforms to requirements and ISO 9001.

  • 2.2 Schedule includes QMS and process audits.

  • 2.3 Takes into consideration the status and the importance of the processes and areas to be audited, as well as the results of previous audits.

  • 2.4 Schedule addresses auditor assignment impartiality to the area assigned.

  • 2.5 Includes audit criteria, scooe, frequency and methods.

  • 3. How are audit results monitored and used for continual improvement? ( ISO 8.2.2)

  • 3.1 Report audit results to management and include in management review.

  • 3.2 Take corrective actions without undue delay to eliminate detected nonconformities and their causes.

  • 3.3 Perform verification follow-up to verify the effectiveness of the corrective actions.

  • 3.4 Maintain internal audit records according to Control of Records.

  • 3.5 Internal audit process metrics are identified, trended, analyzed, and improved towards goals as appropriate ( i.e. performance to schedule, # findings open and closed, aging of open actions, etc.).

  • NOTE: Identify existing metrics on the "Metrics" worksheet

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

8.3 - Control of Non-Conforming Product

  • This section assesses the organization's ability to control nonconforming product.

  • 1. How do you ensure the nonconforming materials / product process addresses the following? ( ISO 8.3, 8.4)

  • 1.1 A documented procedure is required. Defines the controls and related responsabilities and authorities for dealing with nonconforming product.

  • 1.2 Product that does not conform to product requirements is identified and controlled for prevent it from being delivered to the customer.

  • 1.3 Required actions if nonconforming product escapes to the customer is defined.

  • 1.4 Control of nonconforming product process metrics are identified, trended, analyzed, and improved towards goals as appropriate ( i.e. cycle time/aging to disposition decision, time to implement disposition actions, $'s tied up in MRB, MRB participation, scrap levels, rework levels, corrective actions status, etc.)

  • NOTE: identify existing metrics on the "Metrics" worksheet.

  • 2. How are materials identified, segregated, and dispositioned? ( ISO 8.3)

  • 2.1 Nonconforming product is tagged and segregated from good product.

  • 2.2 A cross-functional team (MRB) is used to disposition nonconforming product.

  • 2.3 The team dispositions nonconforming product to either eliminate it, rework, get concession from the customer to use as-is, or scrap.

  • 2.4 Records of the nonconformity and any subsequent actions taken are maintained according to Control of Records.

  • 3. How are Corrective Actions identified and implemented to drive effective improvements? ( ISO 8.3)

  • 3.1 Criteria is defined to determine what Non-conforming materials require formal corrective actions ( i.e. major issues, repeat issues, top issues from analysis, customer impact, high scrap cost, etc.)

  • 3.2 Evidence that Corrective Actions are launched when above criteria is met.

  • 3.3 Corrective action report include determining and documenting any needed containment actions.

  • 3.4 Root Causes sampled demonstrate comprehensive root cause analysis to determine systemic issues ( i.e. 5 why, cause and effect diagram, etc.) / Personnel involved have been through problem solving training ( i.e. 8D)

  • 3.5 Corrective actions are validated as effective ( i.e. physical validation and approval of actions taken, pre-determined monitoring periods, periodic audits of corrective actions, measuring repeat occurences, etc.)

  • 4. How do you ensure that reworked product is verified to ensure that it meets the same requirements as non-reworked product? (ISO 8.3)

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

8.5 - Improvement

  • This section assesses the organization's ability to continually improve the effectiveness of the quality management system.

  • Corrective and Preventive Action.

  • 1. How do you ensure that your corrective action process addresses the following? (ISO 8.5.2)

  • 1.1 Take Corrective Actions to eliminate the causes of nonconformities to prevent recurrence and are appropriate to the effects on the nonconformity.

  • 1.2 Review of corrective actions by management and included in the management review meeting.

  • 1.3 Determine training requirements and train users as required ( process and problem solving tools)

  • 1.4 Evidence that 8D methodology is effectively utilized for any AMAT reported quality defects ( there should be knowledge of the top issue along with corresponding 8D reports).

  • 1.5 Defined process to notify customers when issues are found after the delivery of the product customers. The process should include the risk assessment of customer impact, traceability of suspected materials, and customer notification ( i.e. global recall notice, etc.).

  • 1.6 Corrective action process metrics are identified, trended, analyzed, and improved towards goals as appropriate ( i.e.. # open, # closed, cycle time open / aging, adequacy of closure rates, etc.)

  • NOTE: Identify existing metrics on the "Metrics" worksheet.

  • 2. How does the documented corrective action procedure address the following? ( ISO 8.5.2)

  • 2.1 What corrective actions need to be issued against ( e.g. product, QMS process, suppliers, customers complaints, etc.)

  • 2.2 When corrective actions need to be taken.

  • 2.3 Determine the cause of the nonconformance. Includes problem solving tools that can be used to find root cause ( e.g. 5-Why, fishbone diagram, pareto analysis, etc.)

  • 2.4 Containment actions are defined, documented and implemented when necessary.

  • 2.5 Taking corrective actions with specified time periodnof completion.

  • 2.6 Reviewing the effectiveness of corrective actions taken.

  • 2.7 Maintaining records according to Control of Records.

  • 3. How do you ensure that your preventive action process address the following? ( ISO 8.5.3)

  • 3.1 Determines actions to eliminate the causes of nonconformities to prevent their occurence and are appropriate to the effects of the potential problem.

  • 3.2 Review od preventive actions by management and included in the management review meeting.

  • 4. How does the documented preventive action procedure address the following? (ISO 8.5.3)

  • 4.1 Identifying potential nonconformities and their causes.

  • 4.2 What preventive actions need to be issues against (e.g. product, QMS process, customer complaints, etc.)

  • 4.3 Determining how and when actions need to be taken.

  • 4.4 Reviewing the effectiveness of preventive actions taken.

  • 4.5 Maintaing records according to Controls of Records.

  • Continual Improvement Program.

  • 5. How does the Continual Improvement Program address the following? (ISO 8.5.1)

  • 5.1 Sources for continual improvements should be defined and inputs collected ( i.e. employees suggestions, top issues form analysis of internal and external data, actions from management reviews, strategic planning sessions, etc.)

  • 5.2 Management leads opportunity review, prioritization, and resource allocation process ( i.e. focus on high ROI and customer satistacion opportunities).

  • 5.3 Appropriate improvement methods utilized based on the naturemof the opportunity ( i.e. action item, 8D project, six sigma project, lean project, kaizen event, etc.)

  • 5.4 Personnel have been trained in the use of the selected improvement methodology and improvement tools.

  • 5.5 Management oversight of program implementation and achievement of results towards established goals.

  • 5.6 Rewards and / or recognition program is defined and implemented in support of the continual improvement program.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

2.1.6 - Cycle Time Improvement (CTI)

  • A system is in place to reduce the time it takes to complete a continuous, repetitive work process from start to finish.

  • 1. How does Senior Management focus on cycle time improvement?

  • 1.1 Management establishes clear goals and objectives for cycle time reduction ( i.e. value streams have defined cycle time/ lead time baselines and future state goals for improvement)

  • 1.2 Management has identified owners for cycle time improvement projects ( i.e. Value stream managers, project leaders).

  • 1.3 Targeted cycle time improvements are clearly linked to overall bussiness objectives.

  • 1.4 Cycle time improvement goals, status, and project status are effectively communicated throughout the organization.

  • 1.5 Cycle time improvement success are parts of an overall rewards and recognition system.

  • 1.6 Cycle time improvement progress monitoring & results.

  • NOTE: Identify existing metrics on the "Metrics" worksheet.

  • 2. How are cycle time improvement methodology and training defined and implemented?

  • 2.1 Formal improvement methodologies are identified and utilized to drive cycle time improvement projects ( i.e. Value stream mapping as part of the lean/six sigma methodology, specific methodologies for cycle time as analysis and improvement, bussiness process re-engineering methods, etc)

  • 2.2 Quantifiable method and process is used to evaluate, prioritize, and implement cycle time improvements initiatives ( i.e. Cost/benefit analysis, ROI analysis, etc.)

  • 2.3 Training programs are defined and implemented for cycle time improvement projects participants at a minimum.

  • 2.4 Cycle time improvement methods training has been incorporated into appropriate training curriculums ( training requirements).

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

2.2 BUSINESS SYSTEMS AND ORGANIZATIONAL CAPABILITIES

  • SUB-SECTION:

2.2.4 Material Control System

  • A material control system provides timely flow of materials meeting production requirements. A production control system that provides fabrication, assembly, test and shop load schedules is in use.

  • 1. Do you have a computer based materials management system that addresses the following?

  • 1.1 Sales order entry.

  • 1.2 Purchasing planning (e.g. buyer purchase schedules, order status & alert reports, etc.)

  • 1.3 Manufacturing planning (e.g. Production plans ans priorities, job leveling, sequencing / routing, target ship dates, etc.)

  • 1.4 Order configuration management.

  • 1.5 Capacity planning.

  • 1.6 Capable of providing current and location of materials ( e.g. Ordered, received, inventory, production, allocated, scrapped, shipped, etc.)

  • 2. How are material control parameters established, monitored, analyzed, and acted upon to improve productivity? ( e.g. Inventory turns, inventory levels physical inventory accuracy, cycle count inventory accuracy, scrap levels, etc.)

  • NOTE: identify existing metrics on the "Metrics" worksheet.

  • 3. How do you address capacity planning to include the following?

  • 3.1 Integrated into ERP system ( preferred ).

  • 3.2 Dynamically shows process/ line loading and available capacity.

  • 3.3 Factors in both direct and indirect headcount data into utilization calculations.

  • 3.4 Factors in purchased material lead times and customer's required lead times.

  • 3.5 Ramp up contingency plan exists and is regularly updated to include lead times for resource increases ( manpower, equipment, facilities, etc.)

  • 4. How do you measure and monitor bottlenecks/constraints and implement necessary actions? ( e.g. Demand levels, utilization loading, throughputs, efficiencies, yields, downtime, scrap, cycle time, value stream maps, etc.).

  • 5. How are visual management techniques used to aid with materials controls and management?

  • 5.1 5S status.

  • 5.2 Scheduling and prioritization status.

  • 5.3 Visual signs.

  • 5.4 Pictorial work instructions.

  • 5.5 Production boards with metrics.

  • 5.6 Andon system.

  • Lean Manufacturing.

  • 6. What Lean Concepts and tools training have been implemented?

  • 6.1 Training courses have been established.

  • 6.2 Personnel directly involved with lean concepts and tools application have been trained.

  • 6.3 All personnel have been trained in basic lean concepts.

  • 7. How do you utilize Value Stream Mapping analysis to guide the usage of lean improvement tools for optimum results?

  • 7.1 Product lines / families have been designated owners for Value Stream Mapping Analysis.

  • 7.2 Current state value stream maps are developed and used to determine the biggest opportunities for improvement through the application of lean improvement tools. ( i.e. kaizen events, pull system, li ne design, standarized work, Poka Yoke, SMED, TPM, Pull systems / kanban, FIFO, 5S, Visual Management, etc.)

  • 8. How do you utilize lean methods and tool for continual monitoring and improvement?

  • 8.1 Future state value stream maps and corresponding improvement plans and measurable improvement targets are developed (i.e. lead time reduction, cycle time reduction, inventory turns improvement, product and process quality improvement, cost reduction, productivity/efficiency improvements, set-up time improvements, etc.)

  • 8.2 Management monitors improvement plan implementation status.

  • 8.3 Evidence of a continual improvements over time ( i.e. when future state is achieved a new future state is developed).

  • 8.4 Well defined work cells ( e.g. similar production processes are grouped and arranged to improve work flow productivity and meet takt.)

  • 8.5 Continuous flow principles are incorporated on the factory floor and enable product to move through the value stream one piece (ideally) at a time with little or no inventory or waste.

  • 8.6 Pull systems (customer/supplier Kanban with system signals/pull approach, job kitting) are in place to pull WIP through the factory to meet customer demand.

  • 8.7 Leveling - scheduling systems are used to smooth production requirements, visual job boards highlight problem areas and required production, and flexible worksurface can be re-allocated to address bottlenecks.

  • 8.8 Goals and indicators exist for all appropriate machine areas with a program to continuously reduce setup times.

  • 8.9 Employees suggestion system focused on waste reduction.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

2.2.8 Maintenance of Equipment Used in Production

  • Equipment used in production in maintained in a manner that produces consistent results within specified tolerances.

  • 1. How does the Equipment Maintenance process address the following?

  • 1.1 Preventive maintenance activities are defined for all production equipment and test equipment.

  • 1.2 Methodology is established on how PM frequencies are determined and adjusted.

  • 1.3 Master list that includes all PM activities for all production equipment and teste equipment is established. Includes equipment type, equipment ID, equipment location, PM frequency, scheduled dates for PM's.

  • 1.4 PM tasks are defined via procedure and/or checklist. Checklist is used during PM activity. Values are recorded when checked.

  • 1.5 Records exist to support implementation of all maintenance tasks ( dailies, weeklies, monthlies, etc.)

  • 1.6 PM tasks are at minimum based on equipment manufacturer's suggestion ( i.e. from equipment manuals).

  • 1.7 On demands reports are available and actively used to manage progress (i.e. overdue maintenance tasks report, upcoming maintenance task report).

  • 1.8 Equipment Maintenance process metrics are identified, trended, analyzed, and improved towards goals as appropriate ( i.e. on-time completion of maintenance tasks, equipment downtimes, etc.)

  • NOTE: identify existing metrics on the "Metrics" worksheet.

  • 2. How are statistical methods used for each piece of equipment to measure and improve the effectiveness of the PM program?

  • 2.1 MTBF ( mean time between failure) to analyze maintenance frequency & MTTR ( mean time to repair) to reduce repair cycle time.

  • 3. How does Preventive maintenance training address the following?

  • 3.1 Organization has been trained in TPM principles ( evidence of training records).

  • 3.2 Training program for operators to perform routine maintenance on the equipment they operate has been developed and implemented.

  • 3.3 Training program for maintenance technicians has been developed and implemented.

  • 3.4 Required and completed training is maintained in employees training records.

  • 3.5 Appropriate controls are in place to measure and monitor performance and results if maintenance is outsourced.

  • 4. How is spare parts usage analyzed ans used to set inventory levels?

  • 4.1 MTTR is used to identify top hitters if equipment downtime is due waiting for spare parts.

  • 5. How is production equipment and test fixtures with measurement capability part of the calibration program ( e.g. Temperature, time, pressure, etc.)?

  • 6. Are Equipment manuals and test work instructions (TOMS = test operational method sheets) readily available for all production equipment and test fixtures?

  • 7. What are your contingency plans to handle test fixture breakdowns?

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

2.3 - Statistical Methods

  • Statistical Methods (SM) shall be supported with a formal implementation plan, effectively implementated throughout the organization and reinforced with an appropriate training program.

  • 1. How are statistical methods plans or procedures defined to address the following?

  • 1.1 Monitoring and analyzing product and process quality metrics ( trend charts, variability trending, pareto, etc.)

  • 1.2 As part of manufacturing process design, qualification, and on-going monitoring (i.e. DOE, gage R&R, FMEA, Process Capability Analysis, SPC, etc.)

  • 1.3 Continual Improvement program ( i.e. six sigma projects, CpK improvement projects.)

  • 1.4 Procedure or plan is approved by senior management and identifies roles and responsabilities for deployment and monitoring.

  • 2. How is the statistical methods procedure or plan implemented for monitoring and analyzing product and process quality metrics ( trend charts, control charts, variability trending, pareto, etc.) (ISO 8.2.3, 8.2.4, 8.4) Examples of metrics to be monitored and analyzed;

  • 2.1 Product quality ( i.e. supplier quality, internal product quality including defect levels, scrap, and rework, and customer experienced product quality)

  • 2.2 Process quality ( i.e. manufacturing process yields, testing yields, critical process capabilities, statistical process control or critical to quality parameters).

  • NOTE: Identify product and process quality metrics on the " Metrics" worksheet.

  • 3. If SPC is utilized as a process control, how have run rules been established with operators trained to identify out of control conditions?

  • 3.1 Evidence that appropriate actions are promptly taken.

  • 4. How is the statistical methods procedure or plan implemented in support of manufacturing process controls design and qualification ( i.e. DOE, gage R&R, FMEA, Process Capability Analysis, etc.)?

  • 5. How is the statistical methods procedure or plan implemented in support of the Continual Improvement program ( i.e. six sigma projects, CpK improvement projects)?

  • 6. How is statistical methods training planned and deployed to address the following?

  • 6.1 Statistical methods and tools training requirements are defined to include identification of who needs what training ( i.e. training matrix, training curriculums) / requirements also define refresher training requirements.

  • 6.2 Training materials ( internal and / or external) are available in support of statistical metods/tools training requirements.

  • 6.3 Courses are taught by subject matter experts ( i.e. training are practical experience based, certified in six sigma, qualified external consultants, etc.)

  • 6.4 SM training plans are developed and training completion is monitored ( records, reports, etc.)

  • 6.5 Methods are implemented to validate that SM training is effectively used on the job ( i.e. evaluation of demonstrated use of skills, defined requirements for certifications that include demonstrated use of skills and achievement of results ( i.e. green/black belt certifications), defined requirements to maintain certification levels ( i.e. continued demonstrated use of skills and achievement results), etc.)

  • 7. If applicable, how is the AMAT SSPCM ( Supplier Statistical Process Control Monitoring) tool used for process capability and control monitoring and improvement?

  • 7.1 There is an agreement with AMAT on the product/process characteristics to be monitored via SSPCM.

  • 7.2 Evidence that SSPCM data is all being entered and reviewed in a timely manner.

  • 7.3 Evidence that actions are defined and taken when out of control conditions are present.

  • 7.4 Evidence that process capability improvement projects are defined and implemented when CpK < 1.33.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

3.1 SOFTWARE QUALITY

  • SUB-SECTION:

3.1.1 - Documented Process

  • An approved documented process shall be used to guide the development and maintenance of all sofware that impacts total customer satisfaction.

  • 1. How does your software lifecycle management process address the following?

  • 1.1 Documented process for software lifecycles exists, is approved by management, and is maintained ( i.e. software development cycle has been established showing process from conception to release)

  • 1.2 Evidence supports that appropriate personnel have been trained in the process.

  • 1.3 Organization follows one or more industry standard models/guidelines for software development such as CMMI, Agile, Extreme Programming, etc. Methodology provides context for organization's selected software lifecycli management process.

  • 1.4 A strategy exists to improve the documented process along the SW development model guidelines.

  • 2. How do you address software project management?

  • 2.1 Project Managers are assignated to manage projects throughout the software development lifecycle phases.

  • 2.2 Project plans are developed with specific tasks, dates and milestones that align with lifecycle process ( plans address integration with hardware project plans)

  • 2.3 Status on project plans is updated throughout the project.

  • 2.4 Management level progress monitoring is in place ( execution to plans, achievement of deliverables, etc.)

  • 3. How do you ensure conformance to the software lifecycle process? ( Sample one or more projects/ enhancements and follow it through the development lifecycle with attention to compliance to documented process ( a few areas to sample are making sure all functionality is documented, demonstrated testing to requirements, formal version control and release).

  • 4. How do you use customer feedback for driving continual Improvement?

  • 4.1 Mechanisms are implemented to receive customer feedback on software quality (i.e. bugs, satisfaction feedback, etc.)

  • 4.2 Evidence that feedback is collected, reviewed, acted on, and communication loop closed with customer ( look for standarized means of tracking bugs).

  • 4.3 Evidence that customer satisfaction feedback data is used to measure, trend, analyze, and improve the effectiveness of the software lifecycle process.

  • 5. What software development lifecycle process metrics are identified, trended, analyzed, snd improved towards goals as appropriate? Examples of metrics:

  • 5.1 Performance to milestones, completion of tasks at milestones, testing results, post release changes, field performance, bugs reported, etc.).

  • NOTE: Identify existing metrics on the "Metrics" worksheet.

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

3.1.5 - Configuration Management and Change Control

  • Software shall be maintained under documented plans for configuration management and change control, including installation and customer configuration.

  • 1. How is a standard software configuration management process adopted and followed?

  • 1.1 Used for revision control of the software source code snd executables.

  • 1.2 Larger companies may ise industry standard apps like as ClearCase while smaller companies maynuse open source applications like CVS.

  • 2. How does the software configuration management process include effective change control, software status reporting, version and release management, and software quality approval?

  • 2.1 Effective means the organization has few customer issues due to installation of incorrect software revisionss in product shipment.

  • 2.2 Organization demonstrates the use of configuration management of source code and releases. Test Engineering and / or Quality Engineering approve releases. Organization has dedicated CM personnel.

  • 2.3 Software change control is in place using industry standard feature management software such as Bugzilla, Clearquest, or equivalent.

  • 2.4 Software code is revision controlled using standard revision control packages such as Clearcase, OpenCVs, Visual SourceSafe, or equivalent.

  • 3. How are software releases effectively planned?

  • 3.1 Software project release plans show commit dates and performance to those planned dates are monitored ( i.e. Trending of on-time completions).

  • 3.2 Planning included enhancement and bug fixes allocated to the release with margin for late requiremlents and critical fixes.

  • 4. How do you ensure that software is released only with adequate review and control, including software quality approval?

  • 4.1 Test Engineering or Quality Assurance approval for all releases.

  • 4.2 All releases ( i.e. Major releases, beta uprev's, patches) should have reviews and controls.

  • 5. How do you ensure effective change control and notification procedures are used?

  • 5.1 Stakeholder notification of changes to requirements, schedule, and release.

  • 6. How are changes managed to facilitate the minimal use of resources while maximizing the customer's benefits through "bundling" or combining changes?

  • 6.1 Project plan indicates selection of enhancements and bugs based on bussiness needs, customer issues.

  • 7. How is feedback provided to developers and to marketing to help minimize changes in new products?

  • 7.1 Evidence showing customer input. Issue tracking software records.

  • 7.2 Early, effective customer requirements development should minimize enhancements in the middle of the development lifecycle (i.e. number of enhancements after beta (first customer) release).

  • 8. How are functional areas notified of releases and the requirements, specifications, functions, and capabilities contained in each release?

  • 8.1 Evidence showing integration of all teams in project planning,mmeetings( i.e. Product design engineer close integration with software engineers)

  • 8.2 Early involvement of the test group (QA) early in the development lifecycle.

  • 9. How do you ensure procedures are in place to assure that changes impacting customers, such as installation procedures, installation times, or system configurations, are developed and communicated in advance?

  • 9.1 Release notes or their document describing installation procedure.

  • 9.2 Defined customer advocate that ensures the customer's voice is heard and the needs are effectively communicated through the design team.

VALIDATION SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

PREVIOUS SCORES

  • Management Commitment (MC):

  • Systems Approach (SA):

  • Deployment (D):

  • Results (R):

RESULTS

  • Summary:

  • Strenghts:

  • Observations:

  • Opportunities for Improvement:

SCORING CRITERIA

SCORING CRITERIA

Management Commitment (MC)

  • 0 - No recognition of need.
    1 - Management recognizes need and is discussing requirements.
    2 - Management is involved in determining customer needs.
    3 - Process ownership is identified and resources are commited.
    4 - Management conducts process reviews and removes roadblocks.
    5 - Process review activities are consistent part of the culture and are effective.
    6 - Resources are applied to additional process improvements.
    7 - Management at all levels is fully commited to support customer needs and requirements.
    8 - Strategies, systems and methods are in place for improving present excellence.
    9 - Resources are fully deployed to support documented goals.
    10 - Management is totally committed to customer satisfaction and to the process being an industry benchmark.

Systems Approach (SA)

  • 0 - No evidence of systematic approach.
    1 - An effort is being made to define process requirements.
    2 - Process is defined and improvements consider customer requirements.
    3 - Methodology is in place for process evaluation, with procedure for making process changes.
    4 - Processes and improvements are defined, documented and initiated.
    5 - Customer needs and requirements are well integrated into the systems approach.
    6 - Process is in control and integration is beginning to occur with other applications.
    7 - Process provides effective customer feedback mechanisms with evidence of responsiveness and improvement.
    8 - Fully integrated systems approach throughout.
    9 - Fully integrated, innovative systemic processes anticipate and satisfy customer needs.
    10 - Comparisons are used with other world-class processes to maintain position as an industry benchmark.

Deployment (D)

  • 0 - The process may be defined, but there is no sign of implementation.
    1 - Processes are beginning to be deployed in some key areas.
    2 - Processes are deployed and functioning in some key areas.
    3 - Process users generally understand process requirements and how the process works.
    4 - Processes are deployed and functioning in many key areas.
    5 - Users are completely aware of how process works; are able to provide feedback and improve their work.
    6 - Most key areas are actively using process improvements.
    7 - Mechanisms are in place to continuously review customer feedback and make improvements.
    8 - Processes are robust and stable in all key areas.
    9 - Process is pervasively and consistently applied throughout the areas involved.
    10 - Strong links to customer and supplier processes and systems.

Results (R)

  • 0 - No evidence of results or too early to see results.
    1 - Results are anecdotal (unsubstantiated words).
    2 - Some data are gathered, some analysis is conducted.
    3 - Customer satisfaction data are tracked with some positive results and trends.
    4 - Data shows positive trends with the process compliant to requirements.
    5 - Systems approach demonstrates improvement trends in most key areas.
    6 - Most data show very positive trends.
    7 - Customers are satisfied, all requirements are being met.
    8 - Substantially all data show positive trends.
    9 - Customers are extremely satisfied, sustained results always meet or exceed requirements.
    10 - Is regarded as an industry benchmark with excellent improvement trends.

SIGNOFFS

SIGNOFFS

  • Lead Auditor:

  • Support Auditor:

  • Observer:

  • Area Manager:

  • Auditee:

  • Person
  • Add signature

NON-CONFORMANCES

NON-CONFORMANCES

  • Assign?

  • Non-Conformance

  • NC

METRICS

METRICS

  • Where metrics presented during the audit?

  • Metric

  • Metric
  • Trend: