Hi all!
I has been a while since I posted another article, and as I have been working on a few SCEP/NDES related cases recently, it inspired me to write a quick overview of the communication between Jamf Pro, the managed Apple devices and a SCEP or NDES server.
For this post I’ll mainly focus on the communication and avoid going to deep into the matter of configuring the SCEP/NDES server itself or Jamf Pro as this is highly depending on the specific SCEP setup. I mainly want to highlight a more general overview of how the communication happens between all parties.
As you may know there are 2 main (authentication) techniques to request a certificate from a SCEP (NDES) server:
- By using a Static Challenge
- By using a Dynamic Challenge
And apart from that, Jamf Pro offers 2 strategies in view of how the certificates are going to be requested, each with both static as dynamic challenge option:
- Direct SCEP, where it is the device which is requesting the certificate from the SCEP/NDES server
- Jamf as SCEP proxy, where it is actually Jamf Pro which does the certificate request on behalf of the devices
However, in view of communication there tends to be some confusion from time to time. Hence my inspiration for this blog post. So let’s have a closer look.
Let’s start with the most basic but also less frequent scenario: Direct Scep with a Static Challenge.
First of all, why less frequent, well chances are slim that your security team or SCEP admin will like this strategy very much as using a Static Challenge is maybe not the best choice in view of best practise for this kind of purpose. Nevertheless the option is available so let’s have a look at the communication flow.
When you create a configuration profile in Jamf Pro to use a Static Challenge, without putting Jamf Pro as a Proxy in between, your profile payload would basically look similar to this:
The most important field in view of our topic here would be the Static Challenge which you put in the profile. This challenge would be provided by the SCEP admin and is hard coded in the profile. What happens next is that, when you push this profile to your devices, the profile includes all information to allow the device to request a certificate directly from the SCEP/NDES server.
The profile is installed on the device, the device contacts the SCEP server and provides the static challenge it received in the profile to request a certificate.
This means that your devices need to be able to contact the SCEP/NDES server, which could be problematic if this SCEP server is behind a Firewall. If the devices are already on the internal network, or if the SCEP server URL is publicly available, this would not be a problem, but if not, you may find yourself in a situation where your devices need the certificate to connect to the internal network first (802.1x or VPN) before even being able to reach the SCEP server.
In this case the certificate request would obviously fail:
The order at which things happen in the above scenario is as follows:
- Jamf Pro pushes the configuration profile containing the Static Challenge to the devices.
- The device generates a CSR and sends it to the SCEP server. The private key stays on the device.
- The SCEP server responds with a device certificate.
The above scenario with direct SCEP requests with Static Challenges makes sense I guess, however, things get a little more complicated when we use a Dynamic Challenge with configuration profiles in Jamf Pro. As far as I know MDM does not allow devices to be configured to request a dynamic challenge, so this is why it is actually Jamf Pro which does that on behalf of the devices, regardless of whether Jamf Pro is acting as a SCEP Proxy or not!
This is one of the main sources of confusion I see regularly popping up when troubleshooting SCEP issues, so lets clarify this.
Whenever you use a Dynamic Challenge in a configuration profile the order at which things happen is as follows:
- Jamf Pro server makes standard authenticated HTTPS GET request to Dynamic Microsoft CA URL with data contained in SCEP Payload.
- The SCEP/ NDES server responds with challengePassword. Just like if you would manually navigate to the SCEP admin URL of your SCEP server:
- Jamf Pro builds a unique profile with challenge password obtained from the SCEP/NDES server (as per example above: 7F58B07908B558BB)
- Jamf Pro delivers profile to the device via MDM.
Note: Depending the settings of the SCEP/NDES server there is a limit of simultaneous challenge passwords which can remain pending. MS NDES defaults to 5, which remain valid for 60 minutes (can be changed). This means that if the maximum of pending challenges is reached, without the certs being issues (unique challenge used), additional challenge requests will be blocked until the oldest challenge expires, one of the pending challenges is used to issue a cert or in case of MS NDES, IIS is restarted. This is an important thing to remember in view of the device communication (see below)
- Once the profile is installed on the device (containing the unique challenge) it generates CSR and sends it to the SCEP server. The private key stays on the device.
- The SCEP server responds with a device certificate.
In a flowchart this would look like this:
However, in contrast to our scenario with static challenges, we now have 2 elements in the flow which need to be able to reach out to the SCEP/NDES server:
- Jamf Pro to get a Dynamic Challenge (unlike our scenario with a Static Challenge – hard coded in the profile, Jamf Pro needs to get this challenge first in order to be able to generate a unique profile for each device)
- The devices to request their certificate
If either Jamf Pro or the devices can not reach the SCEP/NDES server, the certificate request will fail or the profile will remain pending (and loop):
The fact that Jamf Pro needs to be able to talk to the SCEP/NDES server is specifically important in case your Jamf Pro server is hosted in JamfCloud (or any other cloud service) while your SCEP/NDES server is hosted internally behind your Firewall !
This is regularly a point of confusion as sometimes I see (failing) configurations where either the SCEP URL or the SCEP admin URL is not reachable from either the devices or JamfCloud respectively.
In general the base FQDN would be the same for both, but important is that Jamf Pro (JamfCloud) needs to be able to contact the SCEP Admin URL to get the challenges!
Now, as discussed in the above scenarios of Direct SCEP, Jamf Pro and/or the devices need to be able to contact the SCEP/NDES server. As a summary so far we have:
Static Challenge: Only the devices need to be able to contact the SCEP server
Dynamic Challenge: Both Jamf Pro and the devices need to be able to contact the SCEP server
But what if you know the devices will not be able to contact the SCEP server as it is hosted internally, behind a Firewall, and the devices need a certificate from SCEP to join the internal network (802.1x or VPN) ?
Chicking or egg situation right? (Sidenote: the egg was first, because dinosaurs already legged eggs and chickens evolved from dinosaurs… )
That brings us to Jamf as SCEP Proxy!
In the PKI settings of Jamf Pro we can configure Jamf Pro to act as a proxy for SCEP, and provide the details like SCEP URL and Challenge type. Here again you can opt for either Static or Dynamic challenge. See flowcharts below for the flow or actions.
The purpose of this feature is to eliminate the requirement for devices to be able to contact the SCEP/NDES server, which changes the order of actions to provide a certificate to your devices as follows:
- In the configuration profile you select “Use the External Certificate Authority settings to enable Jamf Pro as SCEP proxy for this configuration profile“
As you can see you don’t need to provide any SCEP URL or challenge type anymore, as these have been defined already in the Jamf Pro PKI settings. Only the Certificate Subject and other specific settings like SAN, redistribution and retry logic need to be configured.
- When you push the profile to your devices, Jamf Pro will first deliver the profile to the device, in order to instruct the device to create a Certificate Signing Request. As it is Jamf Pro which will request the certificate on behalf of the device (compared to Direct SCEP without Jamf Pro as Proxy, where it is the device requesting the certificate), Jamf Pro needs a CSR to do so.
- The devices generates a CSR and sends it to Jamf Pro. The private key stays on the device.
- If the challenge type has been set to Static (PKI settings in Jamf Pro), Jamf Pro proceeds with submitting the CSR to the SCEP/NDES server, gets the cert and delivers it to the device via MDM.
- If the challenge type has been set to Dynamic (PKI settings in Jamf Pro), Jamf Pro will first get a Dynamic Challenge before submitting the CSR to the SCEP server
Static Challenge with Jamf Pro as proxy:
Here we need to make sure Jamf Pro can reach the SCEP/NDES server to request the certificate:
Dynamic Challenge with Jamf Pro as proxy:
Taking all the above into consideration we can summarise the requirements for the communication with the SCEP/NDES server as follows:
DIRECT SCEP (Jamf not acting as Proxy):
- STATIC CHALLENGE: all devices, regardless of their location need to be able to contact the SCEP/NDES server
- DYNAMIC CHALLENGE: all devices AND Jamf Pro need to be able to contact the SCEP/NDES server
JAMF PRO ACTING AS SCEP PROXY (configured in the PKI settings of Jamf Pro and enabled in the configuration profile):
ONLY Jamf Pro needs to be able to contact the SCEP/NDES server
If both your SCEP as your Jamf Pro server are hosted on-premise I’ll leave the discussion on the network design to you and your network admin, but if you are using JamfCloud, have a look at the required IP addresses you need to allow access from to your SCEP server: https://www.jamf.com/jamf-nation/articles/409/permitting-inbound-outbound-traffic-with-jamf-cloud
Oh, and maybe stating the obvious, but the SCEP URL’s you put in any of the above configurations where JamfCloud needs access to your SCEP server, need to be a publicly reachable and resolvable FQDN. For on-premise an FQDN resolvable from your Jamf Pro server, subject to your DNS settings.
That’s it! As always, if you liked the post, hit the like button, tell your friends about it and leave a comment down below!
Brgds,
TTG
Apple ecosystem enthusiast, geek, tech gadget freak, Belgian living in the Netherlands
Manager Technical Support | Jamf
Hi TTG,
First thanks for this article! 🙂
In my understanding, SCEP and. NDES are handled differently when you want to use Dynamic Challenge in Jamf Pro : you have to choose Dynamic option when talkging to aa “pure” SCEP server, and you have to choose Dynamic-Microsoft CA when talking to an Microsoft AD Certificate Server with NDES addon; NDES is specific to Microsoft with a custom implementation of SCEP. Is this correct ?
Also I was told by Jamf Support last year that Dynamic-Microsoft CA option is not supported on Cloud instances. It can only be used with on premise Jamf Pro servers. Is this still the case?
Thanks for your replies!
Emmanuel
PS: and by the way, chicken came first 😉
https://www.theguardian.com/science/2010/jul/18/chicken-and-egg-conundrum-solved
Hi Emmanuel,
You are correct on the fact that for MS NDES you need Dynamic-Microsoft CA as NDES is indeed MS specific. But Dynamic MS CA is no problem for Cloud.
I’ll check that article on the chickens 🙂
Thanks TTG!! This is extremely helpful in getting this setup.
I have been trouble shooting with JAMF for about a year now and it has been very sporadic in working. Thankfully we have it successfully installed on 90% of our devices. The remaining devices are failing because
“The ‘Wi-Fi Network’ payload could not be installed. The Wi-Fi network either could not be found, or could not be connected. You may need to try to reconnect at a later time.”
The wifi payload in this case is the network settings for computers when they are in the office. Since no one is in the office it is expected that it cant be found and I would suspect would fail on all devices… but its not.
I should also add we are doing Direct SCEP with Dynamic Challenge. We are using Jamf Cloud and have the NDES server public facing with an Azure Web Proxy.
Do you have any insight on when JAMF Pro might be adding Always-on VPN key support for MacOS? It works so well alongside things like the dynamic challenge SCEP (as per this blog) but sucks that it is iOS only. Seems macOS supports it though… (https://developer.apple.com/documentation/devicemanagement/vpn/alwayson)
That would be a question which would need to go via the customer succes team.
I can’t discuss features requests or product changes on this blog.
Is your Wifi-Network payload included in the same profile you’re using to deploy the SCEP payload?
I had been observing intermittent/wonky functionality when using a single profile containing both payloads. ie: the profile would sit at “pending” within the Jamf console for 20-30m, sometimes could be sped up by rebooting the target machine.
It wasn’t until I was attempting to install this profile on on my 11.4 VM that I came across that same “The ‘Wi-Fi Network’ payload could not be installed. The Wi-Fi network either could not be found, or could not be connected. You may need to try to reconnect at a later time.” error in the Jamf console.
I broke the payloads apart into separate profiles and ensured the SCEP payload was getting successfully deployed before attempting to install the Wifi-Network payload and they’re playing along much more nicely now.
Kinda makes sense that you’d need the cert required by the Network payload BEFORE installing it…but we weren’t receiving those errors until the 11.4 VM started whining.
No, should work when combined. I never split it. But of course if you create a chicken or egg situation where you need the SCEP cert to auth on the wifi and the SCEP server can not be reached via the network you are on before you connect to the secure wifi… that will not work indeed. But then Jamf as SCEP proxy could fix that.
Apart from that, not sure if a VM is what you want to use to test such workflows.
I was trying to reply to Rob’s comment above and obviously failed- Jamf as SCEP proxy is on our horizon and will make pain points go away.
Just seemed like Jamf’s attempting to process the network payload before the SCEP one and getting stuck as you mentioned.
Jamf has no say in what the device does or in what order things happen when a profile is installed. If you do direct SCEP by device without Jamf as Proxy, Jamf Pro will request a dynamic challenge for the device, put it in the profile and send it to the device. Jamf’s job done.
Then device will use that challenge and send a csr to the scep server to get a cert and then in your network payload you should define the scep cert as identity cert to connect.
If both Jamf (for the challenge) and the device (for the cert request if jamf is not used as proxy) can reach the scep cert, it should all work.
It’s even how things are designed to be in profile as you select the scep as identity in the network payload.
To me this feels like a VM issue. I’ve stopped using VM’s to test deployments 2 years ago. To much inconsistency.
Anyway, combined profile with scep and network always works for me (if scep can be reached by the device)
I hear you on the inconsistencies with testing with VM’s and since the VM’s are typically accessing the corporate network through host’s my bridged connection seldom do I have them go out and retrieve certificates. But, I was just doing some testing regarding updating our VPN client and which necessitated this.
That being said, I still see real macs in our fleet intermittently take eons to install to install the profile containing the SCEP and Network payloads. The same real mac’s consistently install the profile with just the SCEP payload much faster.
SCEP proxy is on the horizon for us and I’m hoping that ameliorates all of this.
Is it safe to say if you have NDES configured for use with Endpoint Manager, you cannot use that same NDES server for communication with JAMF Pro? The Intune Connector intentionally breaks NDES i think to the point it can do nothing else but communicate to Intune. But I am interested if you or others have different luck.
There are some configs I’ve seen which limits NDES to work with Intune only indeed.
Hi all,
Apologies if this seems trivial, I am fairly new to Jamf and this is one of the first projects I am struggling with.
I am having issues with getting the SCEP Proxy set up on our Jamf instance. I’ve verified that the Jamf server can reach the SCEP server, but we are currently running in to an error:
[aultDelegatingFilterProxy] – Error during authentication
javax.servlet.ServletException: com.jamfsoftware.scep.api.client.ScepClientException: org.jscep.client.ClientException: CA certificate fingerprint could not be verified.
We are using the same configuration with a Dynamic-Microsoft CA challenge as the existing non-proxy profile with no success. I’ve reached out to Jamf and currently have a ticket open with them. They’ve mentioned that a version update would resolve this, however it does not seem to be the case.
Thoughts? I appreciate any help!
Hey! I’m afraid this will have to go via support as it’s too complicated to troubleshoot without seeing all settings, full logs and scep communication with the ca etc. I also do want to interfere with support.
No worries! I figured as much, we just all seem to be scratching our heads a little. I appreciate the post, it’s been super helpful in understanding this all more.
If you put in a fingerprint… remove it. Can’t say for sure but chances are you don’t even need it depending your SCEP and how Jamf Pro’s logic works.
We use the the Static Challenge and the SCEP server is publicly reachable….
The URL is pointing to launching an .exe file, and not sure about the Subject… It is OU format, C=US,O=Company Name, CN=DeviceName (not sure that is correct, but that is what we got from the admin….)
I think it is the certificate as well. They gave us a CSR, and I am pretty sure that is not going to work, we need an actual pem or cer, etc…
I think if Jamf keeps showing “Upload Certificate”, then that is a sign that a valid certificate was not uploaded…..Thanks in advance…
Hey, lot of questions to understand what the setup is here. I’d need to see the full current setup and any errors in Jamf Pro logs. But as this would all go a bit beyond the nature and goal of this blog, please open a support ticket. But that said, no idea what a CSR is doing in the setup of scep via Jamf as Proxy.
Hi, I am also new to Jamf and was trying to solve for an issue we have been having with password sync on the Macs. We currently bind the Macs to AD due to having a cert that allows for WiFi. We want to move from binding but need to give these Macs the cert to get WiFi. I wanted to know if you know of any resources that can help with instruction on setting up the SCEP server where Jamf will be connecting to to request the Certs.
Hey! That a bit of a large topic involving many additional considerations to make such as security and network but this could be a good starting point: https://youtu.be/JOwoMJmgi2g
Or this: https://youtu.be/jn0HTWKubFY
Thank you, excellent article! I have this all working for deployment – quick question related to ongoing management. Regarding revoking certificates and OCSP – there is no provision in Jamf for proxy to OCSP, so that would need to be on-LAN-only, or public facing service in order to allow check-in while off-LAN, correct?
Hey! Thanks. No indeed, there is no functionality to put Jamf Pro in charges of that.
A follow-on question now a year into an active/working deployment aided by your excellent article!
How are people handling renewals of the EAP-TLS identity certificate? Our current method basically rescopes the device (IOS and OS X), redeploying the WiFi and SCEP profile, thus re-issuing the certificate. The issue of course is the ‘chicken-or-the-egg’ issue of the user needing to be connected to an alternative WiFi network (or wired) network during the process since it otherwise whacks the current ‘live’ WiFi profile, breaking the connectivity. Apple has a built-in certificate ‘update’ process (in OS X, don’t know about iOS) that will prompt the user to ‘Update’ the certificate when 15 days out from expire and thus re-issue the SCEP request. That would seem preferable, but does it work on iOS ? And can it be scripted so it doesn’t require user interaction?
Curious what others are doing and what is best practice?
Hey!
The profile in JPRO has the option to, indeed, redistribute the profile:
Redistribute Profile Redistribute the profile automatically when its SCEP-issued certificate is the specified number of days from expiring. Configuring this option adds “$PROFILE_IDENTIFIER” to the Subject field
This indeed will push the profile out again but does not require a manual redeploy.
Reason this is required is because the certificate update process of macOS you are referring to does not work for SCEP.
https://support.apple.com/en-us/HT204836
See which certificates are eligible for automatic renewal
Only ADCertificates delivered as part of a device profile are eligible for automatic renewal.
The following certificates are not eligible and must be renewed manually:
ADCertificate payloads delivered as part of a user profile
Certificates delivered as part of an SCEP payload of any kind
Certificates delivered as part of a profile that contains a mobile device management (MDM) payload
Certificates delivered as part of an over-the-air (OTA) enrollment profile
Thank you for the additional info- we will look into the redistribute profile option. We’ll probably discover soon enough, but doesn’t that still have the unpleasant side effect of breaking the active wifi profile during the redistribution process (eg. your network connection disappears) and is only successful if connected to ‘another network’ until successful redistribution?
Regarding the OS X built-in update process, I actually was successful (Catalina) in testing renewing the certificate issued via SCEP with the press of the update button (testing against on-prem NDES URL, not the Jamf SCEP proxy). The only catch was if a static challenge password was configured. In that case, I needed to blank it out -and the update process then requested a password which I obtained from my NDES server, reissued new cert. We will test using the Jamf SCEP proxy URL as well, which should negotiate the dynamics NDES challenge password.
Hi, I am working on a small school project regarding 802.1x Implementation in Jamf Pro on an iPhone.
Is it possible to use the Static Challenge on a Microsoft CA with the NDES Addon, if I change the Registry on the CA to use a single Password for SCEP?
This is obviously not very secure, but this is only a small test project. I would normally just use the Dynamic-Microsoft CA Setting, but turning that setting on requires me to use the user-initiated Enrollment instead of the automatic Enrollment, which further complicates things beyond the scope/time limit of my project.
Help would be greatly appreciated,
I really liked your article. You have literally translated the situation I am experiencing right now. I have been struggling with this problem for 1 month. I’m not sure I can figure it out.
Hi, Thank you for writing this detailed document it is very helpful. I wanted to understand the dynamic challenge with jamf pro as scep proxy in more detail. According to you the device creates a CSR and sends it to JAMF proxy and then the jamf proxy gets the dynamic challenge and then sends a cert request to the SCEP server , so does jamf proxy create a new CSR with its own private key because it cannot add the dynamic challenge to the already formed CSR right? Also I wanted to understand how does the proxy verify if the requests are actually coming from compliant devices or not?
A CSR does not include at challenge or password at all. The dynamic challenge is part of the authentication, not part of the CSR. Apart from that no, there is no check on compliant devices as such, unless you scope it that way via a scope including only compliant devices.