Everyone knows that the current continuing education (CPE) model in the CPA profession is broken, and while there is a lot of discussion about the future of learning and how the model has to change, the only ones that I know who are making huge strides in enacting the changes is the Indiana CPA Society (INCPAS).



I was excited to be asked to be a part of the advisory board for their CPA Center of Excellence® (CPA COE)—their initiative to shift from hours-based CPE to competency-based CPE. Indiana is leading the United States in accepting the competency-based model, and has already modified its laws to allow competency-based credit, however they are in the middle of rule-making to implement the changes and this is proving harder than they expected.



How is Competency-based CPE Different from Hours-based CPE?



Hours-based CPE is generally earned from participating in instructor-led courses (in-person or via webinar) or by taking self-study courses for which the number of hours required to complete has been determined. Requirements vary from state to state, but in general, most CPAs are required to earn at least 40 hours of CPE each year to maintain our licenses.



The flaw in hours-based CPE is that while a person may participate in an instructor-led course, there is nothing that requires that the person actually learn anything from the course—they simply need to be physically present (not necessarily mentally present). In many courses and conferences, you can often see people doing other work, checking their email, or sometimes even sleeping.

This is because in-person courses assumes being present equals learning. So if the person teaching the course, or the organization hosting the course, doesn’t moderate the participants’ behavior then a person may be in the course, but not actually learning anything.



On the other hand, self-study courses, require that the person taking the course pass a quiz (usually multiple choice) to demonstrate that they’ve read the materials. While this does provide a means of measuring if the person actually read the materials, it actually just tests the person’s ability to recall or find information rather than apply or interpret it. (Note that the same challenge occurs in most academic learning environments as well.)



Then comes competency-based CPE, which then requires that a participant demonstrate that they are able to actually apply the information, demonstrate the skills, or exhibit the behaviors to earn credit for the course. While this sounds great, the ability to actually “test” this type of competency development is much more difficult and complex than simply administering a quiz.



Challenges with Competency-based Learning



The discussion during the CPA COE Advisory Board meeting surfaced some additional challenges with implementing competency-based learning:



• Difficulty determining what is the “right answer.” A key aspect in testing the application of learning is that you can’t use close-ended questions, they must be open-ended an allow the person to demonstrate their understanding and ability to apply the learning. It was pointed out that if you take an open-ended question, and present it to four different experts in the field from which the question was derived, and you’d probably get four different answers—all of which may be right. By their nature, open-ended or reflective questions (those that ask a person to look at how they’ve seen a concept exhibited in their experiences) are difficult to grade.
• Harder to design and grade. Some of difficulty in determining what is the right answer, could potentially be addressed with carefully design questions, but this is easier said than done. Because answers can vary greatly, this often means that they must be graded by a human—and not just any human, a human who can identify what the application of the learning looks like. As the complexity of the material gets higher, the capabilities of the grader also must be higher.
• Higher cost to implement. From the previous bullets, you can see where developing competency-based learning is going to require higher competency (pun intended) in both instructional design and in the subject matter being taught. This results in a higher cost to attract a person with the right education, skills, or background to both design the course and grade the participants’ answers to the course assessments.

I’ve purposely kept the explanations above relatively simplified so as to not have to dive into a bunch of academic theory around learning, but there is a plethora of academic research around “assessment” (how to measure whether learning outcomes have been accomplished), much of which is actually inconclusive on the best way to measure learning and competency.



Should On-The-Job Learning Count Toward Competency?



One of the core values of the CPA profession is “Life-long Learning”. This means that we don’t just learn when we’re in the classroom, but that our careers and ability to deliver high quality services to our clients requires that we be life-long learners—learning and adapting on-the-job as new information becomes available and as the business environment changes.



One of the propositions posed by the INCPAS competency-based credit model is that CPAs should receive credit for on-the-job learning because it represents non-classroom learning and their ability to apply that learning to real-world situations. The discussion raised several points on this:



How do we assess this? The challenges around measuring the amount of actual learning, and who would measure that learning in an on-the-job situation again present themselves in this situation. If academics are still having a hard time performing these types of assessments well, how would a person’s supervisor or even an HR department be able to perform these assessments?

Is it really learning? The other big question is whether learning actually occurred. From a simple perspective, learning should represent an increase in a person’s knowledge or ability to do something. The first few times the learning is applied, there may be increased competency, but those gains are subject to the law of diminishing returns and some research has actually shown that after a certain point in time, competency actually decreases over time because someone that has been doing something for so long has a tendency to become more lax or overconfident in their execution of the competency.

The group didn’t really reach a conclusion on this, but I think that if we can identify the desired parameters and outcomes that demonstrate on-the-job learning, then we may be able to provide credit for on-the-job learning.



A Potential Way to Measure On-The-Job Competency Development



For example, if someone did research on a particular tax issue for a client, then the following parameters could be used:



The issue must be in an area where the person has not previously already demonstrated competency, or the issue must represent an increase in complexity or deeper understanding of the area.

The person must be able to describe or summarize the learning in a way where the learning creates additional explicit knowledge. For example, if they were researching a particular tax issue for a client, then they should be able to describe the key points of the related tax laws or regulations and how these would be applied to interpret the tax handing of a particular situation.

The person must be able to demonstrate repeated application of the learning. So in the example above, they would need to show how they applied the same learning to other tax returns.

The above is based on how I’ve determined if one of my staff has demonstrated their ability to perform the business analysis or project management work that we do well, but I’ve used a tax example above to make it more generally understandable to CPAs.



Arguments Against the Move to a Competency-based CPE Model



One of the questions that arose during the discussion was “are we trying to fix something that isn’t broken?”. Part of the justification to change to competency-based CPE is because it would stop the “cheaters” who are signing up for the in-person learning but doing other work or sleeping through a class. But some of the group felt that cheaters will always find a way to cheat, and someone shared a statistic that the number of people actually cheating the system is actually quite low. So would the changes actually stop the cheaters?



Another way to address the issue is to make the in-person courses more interactive requiring the person to actually participate in the learning. Research around learning also shows that incorporating application or an experiential element into a course increases a person’ retention of what was taught. So this is something that I think should be done anyway to improve the quality of classes being offered. As with competency-based learning, the challenge is getting subject matter experts to also become instructional design experts, or having them work with instructional design experts to revamp their courses. Both options pose resource and inertia issues.



Lastly, I question the value of the proposition of providing credit for on-the-job learning. This is already something that is occurring, and it something integral to providing services to clients, so why do we need to get credit for it? CPAs who are committed to excellence will both ensure that they and their staff are learning on-the-job, and increasing their competence—getting credit for it won’t necessarily increase their competence, and again those that aren’t will find ways to cheat the system.



Is a Shift to Competency-based Credits Really Necessary?



I left the meeting unsure whether our actual compliance requirements had to change. Could we actually address the problems by simply enhancing current learning offerings through increasing the competency of the instructors and improving the design of the courses?



Firms that want to differentiate themselves, both to their clients and their staff (including potential staff), will want to use a competency-based model for both professional development and performance evaluations. This however will require that their HR departments (who are often recruiting and compliance-focused) become more competent in organizational development.



Smaller firms will not be able to do this on their own. I believe this provides a big opportunity for the CPA firm-related associations (state societies, firm alliances, and firm networks) to provide these services to their members. The AICPA has also already provided a competency framework for its Chartered Global Management (CGMA) credential that can be used by accountants working within finance and managerial accounting roles in their organizations.



Opportunities abound with the shift to a competency-based model within the accounting profession. Whether a regulatory change needs to occur to force the adoption of this model within the accounting profession remains to be seen.





This post was originally published on the Improving the World by Inspiring Accountants blog.