We are currently doing some office renovations and it got us thinking. Building courses for the cyber security maturation model shares a lot with building out an office.
We all need a little extra scaffolding at times.
We need a solid foundation, the Testing objectives, domains, and 171 practices of the CMMC-AB, but you also need a plan.
As a Licensed Publishing Partner for CMMC-AB Data Intelligence only utilizes the best instructional designers, university partners and psychometricians across the globe. We train our team on using Pragmatic Instructional Design (McVerry, 2019).
Pragmatic Instructional Design (PID) deploys a seven step process to ensure the highest course quality.
Develop Goals and Concepts
We begin by looking at the source material. In this case the CMMC-AB testing objectives and reference documents such as NIST 800-171. We then apply our 5 X 5 goal setting approach.
What value do we need the cybersecurity expert to bring to their organization in five months? What knowledge should the learner retain in five years? These two questions guide our goal setting for a course
Instructional Design Methods
Pragmatic Instructional Design takes a dialectical approach to adopting multiple perspectives. Meaning opposing ideas can both be true at the same time. There is no one method of instructional design that is better than all the others.
So instead at Data Intelligence Technologies we evaluate our newly formed goals against the context of learning and choose a primary and secondary instructional design theory to guide our development work.
For the cybersecurity maturation model courses we chose to use Understanding by Design and Multimedia Learning Theory.
Identify Course Taxonomy
Next our instructional designers meet with our cybersecurity subject matter experts to create a knowledge and skills tree.
We examine the test objectives and break the body of knowledge down into what a learner needs to know and what a learner needs to do. We recommend this exercise, from Understanding by Design, to all companies where we provide training support.
Take learning to ride a bike. You need to know what a handbrake is (knowledge) and you need to apply various pressure at different speeds (skills). Learning how breaks work requires both. Learning to ride a bike requires learning breaks.
Cognitive scientists break knowledge down into three buckets: declarative (knowing what), procedural (knowing how) and conditional (knowing win). Creating cybersecurity courses requires us to assess across all three.
Create Measures of Growth
Once we know the objectives we switch gears to the assessment. See most people build the class first and then the test. This is backwards...well really it is not backwards. In Understanding By Design methodology you start with the end in mind.
Now that you know where you want to see knowledge growth in your students you have to build a tool to elicit evidence of the growth.
This requires us to develop performance tasks aligned to our objectives and key goals. We also want to apply authentic assessment contexts and this often requires going beyond multiple choice or essay questions.
Some of of CA1 courses for example have assessments embedded in game like play as you work as a new Rookie in the Cyber Cops solving "whodunnit" mysteries
Develop Learning Materials
In the next step of pragmatic instructional design we create a scaffold of activities that builds the skills and knowledge that we will measure on our performance assessment.
The content gets crafted by our instructional designers and our SMEs and we follow a simple navigation. Predictable design leads to improved learning performance. We follow a four phase model of explicit instruction, guided practice, independent practice and reflection.
Content Validity Study
Once we finish the design we send the course out to an independent company to conduct a content validity study. Data Intelligence Technologies does not see the data and always makes the results public.
Our assessments also get rigorous checks of reliability.
You must demand high standards when choosing an LPP. If they can not prove, using industry standards that their course teaches and measures what is syas it will teach in a consistent and reliable way you need to walk away.
Validating tests requires a science. Data Intelligence Technologies uses the best psychometricians in the world to ensure our courses are both valid and reliable.
Pilot Class with Best Practices
We then pilot our class with a group of learners and we iterate on variables that we believe will enhance our pedagogical goal.
We also train our instructors on delivering high quality feedback. All learning derives from learner reflection and teacher feedback. At DIT we utilize the Claim Connect Action (Tepper and Flynn, 2019) method. In CCA our instructors make a claim about user performance against a course objective. They then connect this to evidence elicited from student performance and provide the learner with actionable feedback.]
Choosing the Best Partner
We know there are other Licensed Training Partners and Licensed Publishing Partners in the Marketplace and we want to help implementors and assessors choose the right teammate.
Securing our Nation's data matters more to us than securing your business. We would throw this company away tomorrow if it meant we could flip a switch and harden all the vulnerabilities.
But no switch exists. So we have the CMMC-AB, and we promise to be as transparent as the laws allow. All of our course objectives get an open source license. All third party validation studies and data will be published regardless of results.
We want your help and your trust more than your business, but if you think our Pragmatic Instructional Design approach sounds like a match for you and yours drop us a line.