A compliance program must be tailored to fit the organization. There is no one-size-fits-all program. So how do you know if the program fits and is working? Your program must be assessed to determine if it is truly effective.
You can start with your compliance plan, which should be reviewed periodically. A new regulation or law or new guidelines from regulatory agencies could affect your plan and changes will need to be made. Ask whether what you’ve put in writing (i.e., procedures and policies) actually occurs. Is it working? Could it be improved? The compliance oversight committee should take an active role in this process.
In addition, all compliance-related policies and procedures need to be monitored. A complete review of each policy should be done at least every other year. Annually reviewing them is a best practice, but in most cases doing so is unrealistic. Such a task can obviously become overwhelming, so you may want to consider having a predetermined schedule for reviewing policies and procedures. Certain policies can be reviewed in January, for example, and another batch in April, and so on. Here, too, the compliance committee can help. As you look at compliance-related policies and procedures, consider if they are still necessary. It’s possible a new policy has superseded an existing one. Have circumstances changed to warrant revising a policy or procedure? Are the policies and procedures effective? And as discussed, policies that are written and not followed can lead to trouble. Be sure to evaluate whether all employees are aware of the policies and procedures pertinent to their positions. You can’t expect them to follow policies if they don’t know about them.
Benchmarking against yourself is also a good way to measure program effectiveness, evaluate and support continuous evaluation, and improve the compliance program. Evaluating your current practices compared with other organizations can help to identify standards for achieving best practices. Your annual report provides one regular statistical summation that can be used to develop benchmarking statistics. You might, for example, track and compare the number of educational programs delivered or number of employees trained in a given period. Or compare the number of issues reported and the number of issues later substantiated. Just be sure to collect consistent data so result comparisons are viable. Set goals and objectives for improved performance.
Evaluating for Success
To determine if your efforts at building a compliance program are successful, the following tactics can yield important information:
-
Conduct an annual review of the written compliance program
-
Continually review individual policies and procedures
-
Benchmark against your own statistics
-
Survey and compare
-
Compare your program’s progress to the industry’s
Measuring Effectiveness
The DOJ states, “One hallmark of an effective compliance program is its capacity to improve and evolve.”[1] How can you tell if your program is effective? What is effectiveness? The DOJ discusses evaluating corporate compliance programs and ideas for determining effectiveness in its 2020 update to the Evaluation of Corporate Compliance Programs guidance. A program can generally be considered effective if it includes a basic design infrastructure that includes the FSG seven elements:
1. Written standards of conduct and policies and procedures
2. Designation of a chief compliance officer and other appropriate bodies
3. Effective education and training
-
General training
-
Risk-based training
-
Form and language of training
-
Method of training
-
Examination of how training has impacted operations
4. Audits and evaluation techniques to monitor compliance
5. Establishment of reporting processes and procedures for complaints
6. Appropriate disciplinary mechanisms
7. Investigation and remediation of systemic problems
More specifically, the DOJ advises that the following questions should be considered when evaluating the effectiveness of a compliance program. These questions are:
-
“Is the corporation’s compliance program well designed?”
-
“Is the program being applied earnestly and in good faith?” In other words, is the program adequately resourced and empowered to function effectively?
-
“Does the corporation’s compliance program work” in practice?[2]
The compliance professional should consider these questions and be able to answer them. In terms of design, ask if the structure is what it should be, and is it working? There should be a budget for the compliance program, and the compliance professional should have input on how the resources should be used. Consider whether the authority of the compliance professional has been defined for the organization: Is the professional empowered? Overall, the question remains: Can you as the compliance professional explain how effective the compliance program is?
Effectiveness Measures
Additional effectiveness measures can also be used. As identified by SCCE Compliance 101 educational seminar participants, here are more ideas for measuring effectiveness:
-
Compare issues year to year.
-
Survey employees to gauge compliance culture.
-
Track and trend complaints.
-
Track corrective actions and confirm no reoccurrence of related issues.
-
Review concurrent audits.
-
Compare educational session pre- and post-tests.
-
Track external agencies’ findings, fines, and penalties.
-
Review organizational survey results.
-
Analyze audit results and strength of controls.
-
Ensure compliance has been integrated into organizational discussions.
But how do you achieve effectiveness? One method requires identifying three measures of effectiveness: structure, process, and outcome. Structure refers to the capacity of an organization to provide services, including staffing levels and policies and procedures. Process refers to performance measures on the manner in which business is conducted. And outcome addresses observable, measurable results. One must first identify what the attributes are of an effective compliance program, and only then can you begin to review the program for effectiveness.
Six Steps to Building a Framework for Compliance Effectiveness
1. Identify compliance risk areas.
2. Identify how the organization addresses the identified risk areas by categorizing compliance program elements into structure, process, and outcome measures.
3. Assess the maturity of the compliance program before drawing definitive conclusions about its effectiveness.
4. Evaluate the extent to which structure, process, and outcome measures of effectiveness are viewed as linked by the compliance program.
5. Evaluate the extent to which the compliance program is dynamic and continuously changing in response to internal and external factors.
6. Measure compliance program effectiveness against both guidance from regulatory agencies and organizational goals.
Compliance Program Breaking Points
Given that compliance programs must be tailored to fit an organization and that no two compliance programs are identical, it may be difficult to truly evaluate the effectiveness of your organization’s compliance program. However, organizations can readily identify when their program is suffering and has barriers to effectiveness. The following are common compliance breakdowns that could indicate the need for program modification or enhancement:
-
Compliance officer has inadequate technical skills (auditing, verbal and written communication), knowledge (finance, operations, and legal requirements), compliance vision, and resourcefulness.
-
Lack of financial resources.
-
Lack of commitment from employees, vendors, management, CEO, and board of directors.
-
Compliance officer lacks authority to enforce standards, policies, and procedures.
-
Compliance officer lacks a direct line of communication with the CEO and board of directors.
-
Compliance responsibilities are outsourced to avoid accountability or integration into the organization’s operations.
-
Conflicts of interest and/or compliance officer lacks independence.
-
Conflicts of interest and/or auditors lack independence.
-
Lack of proper dissemination of policies and procedures (or complete absence of policies and procedures).
-
Inaccurate, highly theoretical, non-tailored, and out-of-date policies and procedures.
-
Poor, incorrect, inadequate training content (in general or for the specific audience).
-
Unqualified trainer or train-the-trainer dilution of content.
-
Education sessions too long, overpacked with information, not made to be interesting (monotone trainer, lack of multimedia use), not required, or not frequent enough.
-
Lack of variation in education (training sessions, memos, postings, one-on-one instruction, web-based training, etc.).
-
Education that relies too heavily on online or web-based training programs.
-
Lack of understanding of what should be reported and the obligation to report suspected inappropriate actions.
-
Lack of an open culture and support for retaliation.
-
Lack of anonymous reporting mechanisms or knowledge of the mechanisms for reporting.
-
Fear of retaliation or retaliation itself.
-
Poor follow-through with information communicated or lack of feedback regarding resolution.
-
Disciplinary action plan not communicated or made clear to employees and contractors.
-
Disciplinary action plan not enforced as stated and when necessary.
-
Disciplinary action plan not progressive or fitting for the issue.
-
Disciplinary action plan not determined on a case-by-case basis.
-
Disciplinary action that is not consistent by offense.
-
Insufficient auditing and monitoring schedule (substance, number, frequency), or a schedule that is not followed or not dynamic and able to change to fit new situations
-
Lack of fraud alerts
-
Lack of awareness of industry developments
-
Poorly trained auditors (in auditing techniques or content of audit), uncooperative auditors or auditors not receiving cooperation.
-
Investigations not thorough, comprehensive, or timely.
-
Immediate action not taken to remediate problem.
-
Long-term corrective action plans not put into place.
-
Lack of continued monitoring into areas of proven noncompliance.
-
Poor enforcement of disciplinary guidelines.
Compliance programs should continue to mature over time. You should see growth in your program each time you evaluate its effectiveness The DOJ reminds compliance professionals that evidence is needed showing that revisions to the compliance program utilize lessons learned from investigations, auditing and monitoring results, and other aspects of the compliance program. By incorporating lessons learned from your evaluations, you will continue to see your compliance program evolve.