Today we are going to be taking a look at setting up an Offline Root CA on Windows Server 2016. There are a few serious considerations when setting up a new Root CA. We’ll start from the top with the first serious choice; choosing the hardware the Root CA will be running on. There are a plethora of options when considering hardware for a new Offline Root CA. We want a few specific things;
- We want the Offline Root CA to be survivable. While we will be taking backups regardless of the hardware, if you can afford the expense it makes sense to use real server hardware, specifically for usage with enterprise RAID storage solutions. Booting up your Root CA to realize a single bad disk means you need to restore from backup is not a fun time. Remember, offline means offline. By definition there can be no external monitoring of the Offline Root CA and no chance to catch hardware failures aside from physically checking.
- We want the Offline Root CA to have security features. At the very least we would want our Root CA to have a physical TPM for BitLocker encryption. We may also want to consider a local HSM [Hardware Security Module] such as an nShield Solo HSM or YubiHSM. For many even the cheaper local HSMs will be cost prohibitive. While we are not going to use a Hardware Key Storage Provider for this guide enterprise grade PKI should seriously consider hardware level key protection.
- We want the Offline Root CA to be physically secured. The Offline Root CA must be physically secured before all else. While physical controls are outside of the scope of this guide you should consider who will receive physical access to the Offline Root CA, how they will get that access, and secure storage of credentials required to access the Offline Root CA.
- We don’t need high spec hardware. The scope of work the Offline Root CA is going to be doing is very specific. We do not need excessive RAM, processing power, disk speed, or disk capacity.
- It is important that the device not be networked, ever. We want to eliminate every possible attack vector to the device. This includes any network connectivity, ever. While this does add some tedium to the deployment process you can rest easy knowing your Offline Root CA is secured.
For the purpose of this guide we are going to consider using a laptop as the Offline Root CA, specifically a Lenovo ThinkPad E530c, lovingly referred to as the Stinkpad. Despite my person affection for the device this is not ideal for reasons listed above, the two specific reasons are:
- The laptop has a single point of failure in the hard disk the operating system is stored on.
- The laptop does not have a BitLocker compatible TPM
One of the things using a laptop as the Offline Root CA has going for it is that due to it’s form factor it can be easily stored and secured. While not every business can afford a CoLo they can certainly afford a bank vault or other form of secure storage for a laptop sized device.
Planning for Security
Now that we have determined the hardware for the Offline Root CA we can begin looking at strategies for security. There are a few tasks we would need the Offline Root CA for, specifically
- Signing the Root CRL
- Signing certificate requests for new Issuing CAs
These events should be planned well in advance of execution, including the necessary work to gain physical access to the Offline Root CA as well as the credentials required. While all business cases may be different here would be my plan for my Offline Root CA:
- Root CRLs will be valid for four months, renewal to be planned every three months
- When not in use my Offline Root CA will be stored in a safe deposit box at my local bank (Secure Location A)
- The backup disks for my Offline Root CA will be stored in a separate safe deposit box at a bank other than the one storing the Offline Root CA (Secure Location B)
- The credentials for the Root CA, Root CA Hardware, Secure USB Flash Drive, and Backup disks will be stored in a KeePass database on a separate secure server.
Again, your business case will most likely be different. This is the basic plan we will be following for this deployment.
Pre Imaging Configuration
Hardware Settings
Firstly your Offline Root CA Laptop should have proper security hygiene before your Windows Server 2016 OS is deployed. This is dependent on your hardware but here are few things to consider:
- Ensure the BIOS is updated to the latest secure version available
- Ensure your BIOS is password restricted
- If applicable require BIOS password to boot
- Use UEFI booting if possible, avoid Legacy Boot modes.
- Disable networking at the BIOS level, including any wireless, ethernet, or bluetooth capabilities. If your Offline Root CA hardware has any way to ingress or egress data aside from USB it should be disabled at the hardware level.
Once the mentioned items have been configured and confirmed begin the process of creating your Windows Server 2016 installation media. Download a fresh Windows ISO directly from Microsoft and confirm hash values are correct. It may also be beneficial to download offline Windows Update Rollups to copy on to the installation medium to perform Windows Updates offline, but is not required. During this process you should capture the credentials to store in your secure credential management system. If you do not have a credential management system consider looking into KeePass.
The credentials we have so far are:
- Password to enter Offline Root CA BIOS
- Password to boot Offline Root CA hard disk
- Password for Local Administrator account for the Offline Root CA
Post Imaging Configuration
Change Computer Name
Open an Administrative CMD prompt
Open the SConfig tool by entering the following command
sconfig
Enter 2 and press Enter
Enter the new desired computer name. For this guide the Offline Root CA will be named LAG-ORCA-01.
Once the desired name has been entered press Enter
When prompted to restart the computer click Yes
Disable Networking
Log in to the Offline Root CA as Administrator.
Open an Administrative PowerShell prompt.
In your Administrative PowerShell prompt open your local device manager by entering the following command
devmgmt.msc
For any options under Network adapters right click the name of the device and click Disable. If you are given a warning that the device will stop functioning click Yes.
Creating a Secure USB Flash Drive for the Offline Root CA
We need a way to get data on to and off of the Offline Root CA in a secure way. We can do this by creating a secure USB flash drive that is encrypted with BitLocker. We want to create the secure storage on the CA itself, but we must prepare it on another computer.
Plug the USB Flash Drive to be used as the Secure USB Flash Drive into another Windows Computer.
Open an Administrative CMD Prompt and open the disk part tool by entering the follow command
diskpart
We can search for our flash drive by using the list disk command. In the diskpart terminal enter the following command
list disk
Find the Disk Number associated with your flash drive. In my case I was able to identify my flash drive as Disk 2
Select the flash drive as the current disk by using the following command, replacing the disk number with your own
select disk 2
Remove data from the USB disk by entering the clean command as shown
clean
Once we have confirmed the disk has been cleaned you can remove it from your current computer and plug it in to the Offline Root CA
On your Offline Root CA plug the Secure USB Flash Drive. Open Windows Disk Manager by entering the following command in an Administrative PowerShell prompt
diskmgmt.msc
In the Disk Management Window locate the Secure USB Flash Drive. If necessary delete any volumes by right clicking the volume and selecting Delete Volume…
Once the entire disk space is listed as Unallocated right click the Unallocated block and select New Simple Volume…
Click Next
Accept the default parameters to size the new volume to the entire disk space. Click Next
Assign the default drive letter and click Next
Ensure Format this volume with the following settings is selected and ensure the following values are selected
File System – FAT32
Allocation unit size – Default
Volume Label – ORCA-USB
Perform a quick format – Unchecked
Enable file and folder compression – Unchecked
Once the settings are confirmed click Next
Review the settings in the wizard. Once you are satisfied click Finish
As we are performing an entire disk format to ensure data erasure the process may take some time to complete.
Encrypting the Secure USB
On your Offline Root CA, open File Explorer
Right click the ORCA-USB drive and click Turn on BitLocker
Select Use a password to unlock the drive. Enter and confirm a secure password. Add this password to your KeePass database. Click Next
Click Print the recovery key. Using the Microsoft Print to PDF virtual printer save the recovery PDF as OfflineCASecureFlashDrive.pdf.
Click Next
Ensure Encrypt entire drive is selected and click Next
Ensure Compatible mode is selected and click Next
Click Start Encrypting
Full encryption of the disk may take some time. Do not remove the drive we are encrypting until the process is complete.
Enable OS Disk Bitlocker with PIN
In your Administrative PowerShell prompt open Server Manager by entering the command
servermanager
In the Server Manager window click Add Roles and Features
Click Next
Ensure Role-Based or Feature-Based Configuration is selected and click Next
Confirm you have your Offline Root CA selected and click Next
Do not add any additional roles. Click Next
In the Select Features window ensure Bitlocker Drive Encryption is selected. Accept any added changes and click Add Features.
Click Next
Confirm your installation options and click Install
After the installation completes you will be informed the computer needs to reboot. At this point reboot the server. Once it comes back up log in as the local Administrator and relaunch an Administrative PowerShell prompt.
In your Administrative PowerShell prompt open local group policy configuration by entering the following command
gpedit.msc
Navigate to Computer Configuration > Administrative Templates > Windows Components > BitLocker Drive Encryption > Operating System Drives
Right click Require additional authentication at startup.
Ensure the following fields are as follows:
The GPO is listed as Enabled
Allow Bitlocker without compatible TPM [only check this if you do not have a BitLocker compatible TPM]
Configure TPM startup: Allow TPM
Configure TPM startup PIN: Allow startup PIN with TPM
Configure TPM startup key: Allow startup key with TPM
Configure TPM startup key and PIN: Allow startup key and PIN with TPM
Click Apply
Open File Explorer
Right click your Operating System [C:\] Drive and click Turn on BitLocker.
Click Enter a password
Generate a secure password and save it in your KeePass database. Enter the password and confirm it. Click Next.
Click Print the recovery key. Using the Microsoft Print to PDF virtual printer save the recovery PDF as OfflineCAOSDrive.pdf.
Click Next
When asked how much of the drive to encrypt select Encrypt entire drive
When asked for an encryption mode select New encryption mode and click Next
Ensure the Run BitLocker system check box is checked and click Continue
When prompted to restart your computer click Restart now
Upon rebooting you will be prompted for the password to unlock the drive. Enter your BitLocker password and press Enter
Confirm your operating system boots and you are able to log back in as local Administrator.
Configure Root CA Roles
Install ADCS
While logged in as local Administrator open an Administrative PowerShell prompt.
Use the following command to install the ADCS Server Role
Add-WindowsFeature ADCS-Cert-Authority -IncludeManagementTools
Ensure the role installed sucessfully and that we see the Success Code as True
ADCS Configuration
We can now configure the Offline Root CA ADCS parameters. In your Administrative PowerShell prompt define the following variables:
$CAName – The name you want to give your Root Certification Authority
$CAName = "LaGrassa.org Root CA 01"
$CryptoProvider – The cryptographic provider to store the private keys for your Root CA certificates. If you are using a Hardware Security Module [HSM] you will need to install and configure the HSM on the Root CA before this step. Refer to your HSM documentation for that process as well as the name of your cryptographic storage provider to input in this variable. We are using the default Microsoft software cryptographic provider for the purpose of this guide.
$CryptoProvider = “RSA#Microsoft Software Key Storage Provider”
$HashAlgo – This is the algorithm we are using to generate our Root CA certificate. SHA256 is currently considered a secure industry standard hashing algorithm. Do not use older or less secure hashing algorithms.
$HashAlgo = "SHA256"
$KeyLength – The number (in bits) in our encryption key. Longer does not always necessarily mean better or more secure. 2048 is a currently accepted industry standard for RSA256 key encryption length [As of May 2020].
$KeyLength = "2048"
$YearsValid – The number (in years) our RootCA certificate will be valid for. Typically we want these periods to be short but for Root CAs this is different. We want the certificate to be generates and to bring the server offline, meaning we want a key that is valid essentially in perpetuity. The only time this key should ever be changed is in the event the algorithm used to generate becomes insecure (See SHA1) or the Root CA has been compromised. Either situation will require manual intervention which will require regeneration of the Root CA certificate under new parameters.
$YearsValid = "30"
$DBLoc – When installing ADCS using PowerShell we must define a local database path. Failing to do so will result in an error regarding failed network connectivity as we have no Networking Cards running.
$DBLoc = $(Join-Path $env:SystemRoot "System32\CertLog")
Once our variables are defined we can run
Install-ADCSCertificationAuthority -CACommonName $CAName -CAType StandaloneRootCA -CryptoProviderName $CryptoProvider -HashAlgorithmName $HashAlgo -KeyLength $KeyLength -ValidityPeriod Years -ValidityPeriodUnits $YearsValid -DatabaseDirectory $DBLoc
Confirm you want to run the script by clicking Yes
Confirm the ErrorID returned is 0 and that the installation completed without error.
Our Offline RootCA is a standalone CA not joined to any domain. Despite this there are items to configure in AD to get the full range of tools available to Windows Certificate Services. The options we must configure is as follows:
DSConfigDN – Because we have specific Distinguished Names in our Root CA we need the Root CA to be aware of our domain information to properly publish certificates.
We can configure the registry to update our DN [Distinguished Name] configs.
On your Root CA open PowerShell as an administrator and run:
certutil.exe –setreg ca\DSConfigDN CN=Configuration,DC=lagrassa,DC=org
Be sure to update your DC/LDAP string information unless you are planning a hostile takeover of my homelab.
Once you have updated and ran the command ensure the output shows success.
Define CDP Location – We need to define where computers can check certificate revocation status, or CRLs. This is done through Active Directory or through clear web traffic [HTTP on port 80. [Note: CRLs MUST be published accessible by HTTP on port 80. using HTTPs on port 443 will not work] You must configure a web server to host the CRLs under a DNS A record that will not change. You do not need a web server at this point but will need one to complete your PKI setup.
For the purpose of this guide I have a web server named ipki.lagrassa.org that will serve as my CDP. Be sure to update this when you run your command.
In an Administrative PowerShell prompt enter
certutil -setreg CA\CRLPublicationURLs "1:C:\Windows\system32\CertSrv\CertEnroll\%3%8%9.crl \n10:ldap:///CN=%7%8,CN=%2,CN=CDP,CN=Public Key Services,CN=Services,%6%10\n2:http://ipki.lagrassa.org/CertEnroll/%3%8%9.crl"
Update AIA (Authority Information Access) – This information is included in each certificate and defined where a computer or application can check the Issuing CA’s certificate. We can define this to be the same path as the CDP. Once again make sure to update the URL defined. In an Administrative PowerShell prompt run:
certutil -setreg CA\CACertPublicationURLs "1:C:\Windows\system32\CertSrv\CertEnroll\%1_%3%4.crt\n2:ldap:///CN=%7,CN=AIA,CN=Public Key Services,CN=Services,%6%11\n2:http://ipki.lagrassa.org/CertEnroll/%1_%3%4.crt"
Confirm the output is successful.
Setting CA Time Limits – Because the Root CA will only be issuing certificates to Issuing CAs we need to configure longer time periods for issued certificates. We can configure this using CertUtil. In an Administrative PowerShell prompt enter the following commands:
certutil -setreg ca\ValidityPeriod "Years"
certutil -setreg ca\ValidityPeriodUnits 5
This will set certificates issued from the Root CA to be valid for 5 years.
We will also need to configure CRL time limits. Here are the example settings we will be using. Please note; since this CA is configured to be offline the Delta CRL should be disabled.
Certutil -setreg CA\CRLPeriodUnits 13
Certutil -setreg CA\CRLPeriod "Weeks"
Certutil -setreg CA\CRLDeltaPeriodUnits 0
Certutil -setreg CA\CRLOverlapPeriodUnits 6
Certutil -setreg CA\CRLOverlapPeriod "Hours"
Confirm the output is successful and restart the Certificate Services service using PowerShell.
Get-Service CertSVC | Restart-Service
Generating the first CRL – We are now ready to generate our first CRL. This can be done simply by using the following command in an administrative PowerShell prompt:
certutil -crl
Verify the the command completed successfully and check the following directory:
C:\Windows\System32\CertSrv\CertEnroll
We should see two items created; a CRL for the Root CA and the Root CA Certificate itself.
Connect and unlock your BitLocked USB flash drive created at the start of this guide. Copy the Root CA Certificate and the CRL to the flash drive, then gracefully eject it.
Publish the Root CA Certificate and CRL in AD – We are now ready to distribute the Root CA certificate. Log on to a Domain Controller with at least Domain Admin credentials and copy the Root CA Certificate as well as the CRL to the Domain Controller.
To import the Root CA certificate open an Administrative Powershell prompt and use the following command:
certutil –f –dspublish "PATH\TO\Cert.crt" RootCA
Verify the output as successful and move on to the CRL:
certutil –f –dspublish "PATH\TO\CRL.crl"
Configure Offline Root CA Backups
We now have a functioning Offline Root CA, albeit with no Issuing CAs listed under it. Let’s finish out the configuration by setting up our backups. For this guide I am recommending two storage media capable of holding multiple server backups that will be encrypted with BitLocker. These can be simple USB external hard drives. They will be referred to as Backup Drive A and Backup Drive B.
Install the Windows Server Backup Role
Install the Windows Server Backup feature by entering the following command in an Administrative PowerShell prompt
Install-WindowsFeature -Name Windows-Server-Backup
Create your Backup Storage Media
On a computer other than the Offline Root CA plug Backup Drive A. Open Windows Disk Manager by entering the following command in an Administrative PowerShell prompt
diskmgmt.msc
In the Disk Management Window locate Backup Drive A. If necessary delete any volumes by right clicking the volume and selecting Delete Volume…
Once the entire disk space is listed as Unallocated right click the Unallocated block and select New Simple Volume…
Click Next
Accept the default parameters to size the new volume to the entire disk space. Click Next
Assign the default drive letter and click Next
Ensure Format this volume with the following settings is selected and ensure the following values are selected
File System – NTFS
Allocation unit size – Default
Volume Label – Offline CA Backup Drive A
Perform a quick format – Unchecked
Enable file and folder compression – Unchecked
Once the settings are confirmed click Next
Review the settings in the wizard. Once you are satisfied click Finish
As we are performing an entire disk format to ensure data erasure the process may take some time to complete, especially on HDD storage media.
Run a virus scan like how we did for the Secure USB Flash Drive. Once the drive has been confirmed as virus-free disconnect the drive.
Repeat this process for Offline CA Backup Drive B
Encrypt your Backup Storage Media
Once both external backup media have been formatted entirely we can begin the process of encrypting them with BitLocker. This will be very similar to how we encrypted the OS volume.
Open File Explorer
Right click the Offline CA Backup Drive A and click Turn on BitLocker
Select Use a password to unlock the drive. Enter and confirm a secure password. Add this password to your KeePass database. Click Next
Click Print the recovery key. Using the Microsoft Print to PDF virtual printer save the recovery PDF as OfflineCABackupDriveA.pdf.
Click Next
Ensure Encrypt entire drive is selected and click Next
Ensure Compatible mode is selected and click Next
Click Start Encrypting
Full encryption of the disk may take some time, especially for HDD based storage mediums. Do not remove the drive we are encrypting until the process is complete.
Repeat this process for Offline CA Backup Drive B
Perform Initial Offline Root CA Windows Backup
Now that we have secure media in which to store our Offline Root CA operating system backups we can take the first backup.
Open Server Manager. This can be done by entering the following command in an Administrative PowerShell prompt
servermanager
Click Tools > Windows Server Backup
In the Windows Server Backup MMC Pane click Local Backup
Allow the backup MMC pane to populate if needed, then right click Local Backup and click Backup Once…
Ensure Different options is selected and click Next
Ensure Full server (recommended) is selected and click Next
Ensure Local drives is selected and click Next
Ensure the Backup destination is listed as Offline CA Backup Drive A and click Next
If you receive a warning that the selected volume is also included in the list of items to back up you should exclude the volume by clicking OK
Confirm the backup settings are correct and click Backup
The Windows Server Backup wizard will run. Ensure the backup Status is Completed and there are no errors.
Repeat this process for Offline CA Backup Drive B
Be sure to test the backups before you need them.
Store the Gathered BitLocker Key Securely
During the process of this guide we should have four PDFs created that contain the recovery codes for our BitLocked storage media. We should have:
- OfflineCABackupDriveA.pdf
- OfflineCABackupDriveB.pdf
- OfflineCAOSDrive.pdf
- OfflineCASecureFlashDrive.pdf
Copy these items to the Secure USB Flash Drive and move them to another computer with access to your KeePass database.
Once the keys have been moved from the Offline Root CA you can delete them and turn Offline Root CA off.
If you have a different method to securely store these PDFs you may consider that an option. For our example I recommend opening the PDFs and manually copying the data to individual KeePass database items. Once the recovery codes are confirmed as safely stored in KeePass delete the PDF files.
Depending on your application used to read the PDFs you may want to clear any local cached data to prevent anyone from being able to read or recover the PDFs holding the recovery keys.
You may also forego digital storage of these backup keys entirely. In such a case the information from the PDFs can be written down manually (do not print them) and verified. Two copies of the document should be created and stored in seperate secure locations.
Physically Securing the Hardware
At this point we have all of our hardware configured. We can move the items as listed:
Secure Location A’s Items
- Offline Root CA Laptop
- Offline Root CA Backup Drive A
Secure Location B’s Items
- Offline Root CA Backup Drive B
Office Storage
- Secure USB Flash Drive
Interaction Strategy
Single Engineer
We would now want to discuss the basic plan for how work on the Offline Root CA would actually be performed. We will use an example of a single engineer needing to generate a new Root CA CRL and perform a regular backup.
- Engineer requests access to read the password for the Secure Flash Drive, Offline CA Backup Drive A, and Offline CA Backup Drive B, Offline Secure CA OS Disk, and Offline Secure CA BIOS Boot Password
- Engineer requests physical access to Secure Location A, and Secure Location B.
- Engineer takes the Secure USB Flash Drive stored in their office and moved to Secure Location B.
- Engineer takes the Offline Root CA Backup Drive B with them to Secure Location A.
- Engineer logs in to the Offline Root CA using the credentials they checked out.
- Engineer performs CRL signing task, copying the new CRL to the Secure USB Flash Drive.
- Engineer performs a new backup on Offline Root CA Backup Drive A, then repeats the process for Offline Root CA Backup Drive B.
- Engineer shuts the Offline Root CA down. They leave the Offline Root CA Backup Drive A in Secure Location A.
- Engineer travels back to Secure Location B and securely stores Offline Root CA Backup Drive B
- Engineer travels back to office and publishes the CRL generated from the Offline Root CA.
Multiple Engineers
With more engineering staff available the process of interacting with the Offline Root CA can be made more secure by removing the ability for any one person to have full control over the processes required to interact with the Offline Root CA. For example you could follow the same process as listed above but require Engineer A to only check out the BIOS boot password and allowing Engineer B to only check out the BitLocker OS Drive password while forbidding either from ever being able to check out the other credentials. Delegating credentials this way would require both engineers to work in tandem with approval from a third party that manages the physical access to the devices. This also creates a situation where no one person ever works alone on the Offline Root CA, making it much harder for a single rouge employee to compromise the PKI.