Allow and control sending UTF-8 emails in the default email sender impl

Closes #41023

Signed-off-by: rmartinc <rmartinc@redhat.com>
Signed-off-by: Alexander Schwartz <aschwart@redhat.com>
Co-authored-by: Alexander Schwartz <aschwart@redhat.com>
(cherry picked from commit 949ef35a3bda916b24763c435033258a84ba8596)
This commit is contained in:
Ricardo Martin 2025-08-15 12:43:38 +02:00 committed by Marek Posolda
parent 3cb390d188
commit 9f653d7e64
12 changed files with 262 additions and 18 deletions

View File

@ -69,6 +69,16 @@ Auth Token Client Secret::
Only needed when *Authentication Type* 'token' is selected.
Supply the *Auth Client Secret* that authenticates the client to fetch a token from the *Auth Token URL*. The value of the *Auth Client Secret* field can refer a value from an external <<_vault-administration,vault>>.
Allow UTF-8::
Enable to UTF-8-encode email address when sending them to the server. This should only be enabled if the mail server supports UTF-8 via the SMTPUTF8 extension. If disabled, domain names containing non-ASCII characters will be encoded using punycode, and addresses containing non-ASCII characters in the local part of the address will return an error.
+
If you do not enable this option, take additional measures to prevent non-ASCII characters in users' email addresses:
+
--
. Verifying that no email addresses of existing users have non-ASCII characters in the local part of the email address.
. Updating the validation of email addresses to prevent non-ASCII characters in the local part of the email address, for example, by adding a regex pattern validation in the user profile for the email address field similar to `\p&#123;ASCII&#125;*@.*` with an error message similar to `Local part of the address must contain only ASCII characters`.
--
ifeval::[{project_community}==true]
== XOAUTH2 email configuration with third-party vendors

View File

@ -0,0 +1,21 @@
// ------------------------ Notable changes ------------------------ //
== Notable changes
Notable changes where an internal behavior changed to prevent common misconfigurations, fix bugs or simplify running {project_name}.
=== UTF-8 management in the email sender
Since this release, {project_name} adds a new option `allowutf8` for the realm SMTP configuration (*Allow UTF-8* field inside the *Email* tab in the *Realm settings* section of the Admin Console).
For more information about email configuration, see the link:{adminguide_link}#_email[Configuring email for a realm] chapter in the {adminguide_name}.
Enabling the option encodes email addresses in UTF-8 when sending them, but it depends on the SMTP server to also supports UTF-8 via the SMTPUTF8 extension.
If *Allow UTF-8* is disabled, {project_name} will encode the domain part of the email address (second part after `@`) using punycode if non-ASCII characters are used, and will reject email addresses that use non-ASCII characters in the local part.
If you have an SMTP server configured for your realm, perform the following migration after the upgrade:
* If your SMTP server supports SMTPUTF8:
. Enable the *Allow UTF-8* option.
* If your SMTP server does not support SMTPUTF8:
. Keep the *Allow UTF-8* option disabled.
. Verify that no email addresses of users have non-ASCII characters in the local part of the email address.
. Update the validation of email addresses to prevent allow non-ASCII characters in the local part of the email address, for example, by adding a regex pattern validation in the user profile for the email address field similar to `\p&#123;ASCII&#125;*@.*` with an error message similar to `Local part of the address must contain only ASCII characters`.

View File

@ -1,6 +1,10 @@
[[migration-changes]]
== Migration Changes
=== Migrating to 26.2.8
include::changes-26_2_8.adoc[leveloffset=2]
=== Migrating to 26.2.6
include::changes-26_2_6.adoc[leveloffset=2]

View File

@ -40,6 +40,7 @@ displayName=Display name
applyToResourceTypeHelp=Specifies if this permission should be applied to all resources with a given type. In this case, this permission will be evaluated for all instances of a given resource type.
cibaIntervalHelp=The minimum amount of time in seconds that the CD (Consumption Device) must wait between polling requests to the token endpoint. If set to 0, the CD must use 5 as the default value according to the CIBA specification.
envelopeFrom=Envelope from
allowutf8=Allow UTF-8
eventTypes.UPDATE_TOTP.name=Update totp
updateCibaError=Could not update CIBA policy\: {{error}}
policyUrl=Policy URL
@ -2719,6 +2720,7 @@ mappingCreatedError=Could not create mapping\: '{{error}}'
deleteClientPolicyProfileConfirmTitle=Delete profile?
passwordPoliciesHelp.forceExpiredPasswordChange=The number of days the password is valid before a new password is required.
envelopeFromHelp=An email address used for bounces (optional).
allowutf8Help=Enable to allow UTF-8 characters in the local part of the email address. This should only be enabled if the mail server supports UTF-8 via the SMTPUTF8 extension. If disabled, domain names containing UTF-8 characters will be encoded using punycode, and addresses containing UTF-8 characters in the local part of the address will return an error.
passwordPoliciesHelp.upperCase=The number of uppercase letters required in the password string.
policyDeletedError=Could not remove the resource {{error}}
key=Key

View File

@ -302,6 +302,16 @@ export const RealmSettingsEmailTab = ({
)}
</>
)}
<SwitchControl
name="smtpServer.allowutf8"
label={t("allowutf8")}
labelIcon={t("allowutf8Help")}
data-testid="smtpServer.allowutf8"
defaultValue=""
labelOn={t("enabled")}
labelOff={t("disabled")}
stringify
/>
<Controller
name="smtpServer.debug"
control={control}

View File

@ -134,7 +134,7 @@ public class RealmImportTest extends BaseOperatorTest {
k8sclient.getKubernetesSerialization().registerKubernetesResource(KeycloakRealmImport.class);
K8sUtils.set(k8sclient, getClass().getResourceAsStream("/example-realm.yaml"), obj -> {
KeycloakRealmImport realmImport = (KeycloakRealmImport) obj;
realmImport.getSpec().getRealm().setSmtpServer(Map.of("port", "${MY_SMTP_PORT}", "host", "${MY_SMTP_SERVER}"));
realmImport.getSpec().getRealm().setSmtpServer(Map.of("port", "${MY_SMTP_PORT}", "host", "${MY_SMTP_SERVER}", "from", "admin@keycloak.org"));
realmImport.getSpec().setPlaceholders(Map.of("MY_SMTP_PORT", new Placeholder(new SecretKeySelectorBuilder().withName("keycloak-smtp-secret").withKey("SMTP_PORT").build()),
"MY_SMTP_SERVER", new Placeholder(new SecretKeySelectorBuilder().withName("keycloak-smtp-secret").withKey("SMTP_SERVER").build())));
return realmImport;

View File

@ -32,4 +32,11 @@ public interface EmailSenderProvider extends Provider {
}
void send(Map<String, String> config, String address, String subject, String textBody, String htmlBody) throws EmailException;
/**
* Validates configuration for the SMTP sender.
* @param config The configuration to test
* @throws EmailException If some error is found
*/
void validate(Map<String, String> config) throws EmailException;
}

View File

@ -17,13 +17,14 @@
package org.keycloak.email;
import jakarta.mail.internet.MimeUtility;
import org.jboss.logging.Logger;
import org.keycloak.common.enums.HostnameVerificationPolicy;
import org.keycloak.models.KeycloakSession;
import org.keycloak.models.UserModel;
import org.keycloak.services.ServicesLogger;
import org.keycloak.truststore.JSSETruststoreConfigurator;
import org.keycloak.utils.EmailValidationUtil;
import org.keycloak.utils.SMTPUtil;
import jakarta.mail.Address;
import jakarta.mail.MessagingException;
@ -36,6 +37,7 @@ import jakarta.mail.internet.InternetAddress;
import jakarta.mail.internet.MimeBodyPart;
import jakarta.mail.internet.MimeMultipart;
import jakarta.mail.internet.MimeMessage;
import jakarta.mail.internet.MimeUtility;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocketFactory;
import java.io.UnsupportedEncodingException;
@ -74,16 +76,20 @@ public class DefaultEmailSenderProvider implements EmailSenderProvider {
@Override
public void send(Map<String, String> config, String address, String subject, String textBody, String htmlBody) throws EmailException {
Session session = Session.getInstance(buildEmailProperties(config));
final boolean allowutf8 = isAllowUTF8(config);
final String convertedAddress = checkUserAddress(address, allowutf8);
final String from = checkFromAddress(config.get("from"), allowutf8);
Message message = buildMessage(session, address, subject, config, buildMultipartBody(textBody, htmlBody));
Session session = Session.getInstance(buildEmailProperties(config, from));
Message message = buildMessage(session, convertedAddress, from, subject, config, buildMultipartBody(textBody, htmlBody));
try(Transport transport = session.getTransport("smtp")) {
EmailAuthenticator selectedAuthenticator = selectAuthenticatorBasedOnConfig(config);
selectedAuthenticator.connect(this.session, config, transport);
transport.sendMessage(message, new InternetAddress[]{new InternetAddress(address)});
transport.sendMessage(message, new InternetAddress[]{new InternetAddress(convertedAddress)});
} catch (Exception e) {
ServicesLogger.LOGGER.failedToSendEmail(e);
@ -91,7 +97,13 @@ public class DefaultEmailSenderProvider implements EmailSenderProvider {
}
}
private Properties buildEmailProperties(Map<String, String> config) throws EmailException {
@Override
public void validate(Map<String, String> config) throws EmailException {
// just static configuration checking here, not really testing email
checkFromAddress(config.get("from"), isAllowUTF8(config));
}
private Properties buildEmailProperties(Map<String, String> config, String from) throws EmailException {
Properties props = new Properties();
if (config.containsKey("host")) {
@ -137,9 +149,9 @@ public class DefaultEmailSenderProvider implements EmailSenderProvider {
props.setProperty("mail.smtp.from", envelopeFrom);
}
String from = config.get("from");
if (from == null) {
throw new EmailException("No sender address configured in the realm settings for emails");
final boolean allowutf8 = isAllowUTF8(config);
if (allowutf8) {
props.setProperty("mail.mime.allowutf8", "true");
}
// Specify 'mail.from' as InternetAddress.getLocalAddress() would otherwise do a InetAddress.getCanonicalHostName
@ -150,12 +162,8 @@ public class DefaultEmailSenderProvider implements EmailSenderProvider {
return props;
}
private Message buildMessage(Session session, String address, String subject, Map<String, String> config, Multipart multipart) throws EmailException {
private Message buildMessage(Session session, String address, String from, String subject, Map<String, String> config, Multipart multipart) throws EmailException {
String from = config.get("from");
if (from == null) {
throw new EmailException("No sender address configured in the realm settings for emails");
}
String fromDisplayName = config.get("fromDisplayName");
String replyTo = config.get("replyTo");
String replyToDisplayName = config.get("replyToDisplayName");
@ -224,6 +232,10 @@ public class DefaultEmailSenderProvider implements EmailSenderProvider {
return "true".equals(config.get("ssl"));
}
private static boolean isAllowUTF8(Map<String, String> config) {
return "true".equals(config.get("allowutf8"));
}
private static boolean isDebugEnabled(Map<String, String> config) {
return "true".equals(config.get("debug"));
}
@ -236,6 +248,42 @@ public class DefaultEmailSenderProvider implements EmailSenderProvider {
return "token".equals(config.get("authType"));
}
private static String checkUserAddress(String address, boolean allowutf8) throws EmailException {
final String convertedAddress = convertEmail(address, allowutf8);
if (convertedAddress == null) {
throw new EmailException(String.format("Invalid address '%s'. If the address contains UTF-8 characters in the local part please ensure the SMTP server supports the SMTPUTF8 extension and enable 'Allow UTF-8' in the email realm configuration.", address));
}
return convertedAddress;
}
private static String checkFromAddress(String from, boolean allowutf8) throws EmailException {
final String covertedFrom = convertEmail(from, allowutf8);
if (from == null) {
throw new EmailException(String.format("Invalid sender address '%s'. If the address contains UTF-8 characters in the local part please ensure the SMTP server supports the SMTPUTF8 extension and enable 'Allow UTF-8' in the email realm configuration.",
from));
}
return covertedFrom;
}
private static String convertEmail(String email, boolean allowutf8) throws EmailException {
if (!EmailValidationUtil.isValidEmail(email)) {
return null;
}
if (allowutf8) {
// if allowutf8 the extension will manage both parts
return email;
}
// if no allowutf8, do the IDN conversion over the domain part
final String convertedEmail = SMTPUtil.convertIDNEmailAddress(email);
if (convertedEmail == null || !convertedEmail.chars().allMatch(c -> c < 128)) {
// now if there are non-ascii characters, we should send an error
return null;
}
return convertedEmail;
}
protected InternetAddress toInternetAddress(String email, String displayName) throws UnsupportedEncodingException, AddressException, EmailException {
if (email == null || "".equals(email.trim())) {

View File

@ -16,6 +16,7 @@
*/
package org.keycloak.services.managers;
import jakarta.ws.rs.BadRequestException;
import jakarta.ws.rs.ClientErrorException;
import jakarta.ws.rs.core.Response;
import org.keycloak.Config;
@ -69,7 +70,9 @@ import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Optional;
import org.keycloak.email.EmailException;
import org.keycloak.utils.ReservedCharValidator;
import org.keycloak.utils.SMTPUtil;
import org.keycloak.utils.StringUtil;
/**
@ -555,6 +558,7 @@ public class RealmManager {
ReservedCharValidator.validate(rep.getRealm());
ReservedCharValidator.validateLocales(rep.getSupportedLocales());
ReservedCharValidator.validateSecurityHeaders(rep.getBrowserSecurityHeaders());
SMTPUtil.checkSMTPConfiguration(session, rep.getSmtpServer());
realm.setName(rep.getRealm());
// setup defaults
@ -668,6 +672,8 @@ public class RealmManager {
session.clientPolicy().updateRealmModelFromRepresentation(realm, rep);
fireRealmPostCreate(realm);
} catch (EmailException e) {
throw new BadRequestException(e.getMessage());
} finally {
session.getContext().setRealm(currentRealm);
}

View File

@ -56,6 +56,7 @@ import org.keycloak.common.Profile;
import org.keycloak.common.VerificationException;
import org.keycloak.common.util.PemUtils;
import org.keycloak.email.EmailAuthenticator;
import org.keycloak.email.EmailException;
import org.keycloak.email.EmailTemplateProvider;
import org.keycloak.events.EventQuery;
import org.keycloak.events.EventStoreProvider;
@ -111,6 +112,7 @@ import org.keycloak.storage.StoreSyncEvent;
import org.keycloak.utils.GroupUtils;
import org.keycloak.utils.ProfileHelper;
import org.keycloak.utils.ReservedCharValidator;
import org.keycloak.utils.SMTPUtil;
import java.io.InputStream;
import java.security.cert.X509Certificate;
@ -474,6 +476,13 @@ public class RealmAdminResource {
throw ErrorResponse.error(e.getMessage(), Status.BAD_REQUEST);
}
try {
SMTPUtil.checkSMTPConfiguration(session, rep.getSmtpServer());
} catch (EmailException e) {
logger.error(e.getMessage(), e);
throw ErrorResponse.error(e.getMessage(), Status.BAD_REQUEST);
}
try {
if (!Constants.GENERATE.equals(rep.getPublicKey()) && (rep.getPrivateKey() != null && rep.getPublicKey() != null)) {
try {

View File

@ -0,0 +1,71 @@
/*
* Copyright 2025 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.keycloak.utils;
import java.net.IDN;
import java.util.Map;
import org.keycloak.email.EmailException;
import org.keycloak.email.EmailSenderProvider;
import org.keycloak.models.KeycloakSession;
/**
* SMTP utility methods.
*
* @author rmartinc
*/
public class SMTPUtil {
private SMTPUtil() {
// static helper class
}
/**
* Validates the configuration using the email sender provider.
*
* @param session The keycloak session to use
* @param config The configuration to validate
* @throws EmailException If some error is found in the configuration
*/
public static void checkSMTPConfiguration(KeycloakSession session, Map<String, String> config) throws EmailException {
if (config == null || config.isEmpty()) {
return;
}
final EmailSenderProvider sender = session.getProvider(EmailSenderProvider.class);
sender.validate(config);
}
/**
* Converts an email address to its ASCII representation using punycode
* (IDN.toASCII) for the domain part. The local part is not modified.
*
* @param email The email to convert
* @return The converted email or null (if IDN.toASCII throws an exception)
*/
public static String convertIDNEmailAddress(String email) {
final int idx = email == null ? -1 : email.indexOf('@');
if (idx < 0) {
return email;
}
try {
return email.substring(0, idx) + '@' + IDN.toASCII(email.substring(idx + 1));
} catch (IllegalArgumentException e) {
return null;
}
}
}

View File

@ -24,11 +24,14 @@ import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import org.keycloak.admin.client.Keycloak;
import org.keycloak.admin.client.resource.RealmResource;
import org.keycloak.admin.client.resource.UserResource;
import org.keycloak.models.AdminRoles;
import org.keycloak.models.Constants;
import org.keycloak.representations.AccessToken;
import org.keycloak.representations.idm.RealmRepresentation;
import org.keycloak.representations.idm.UserRepresentation;
import jakarta.mail.Address;
import jakarta.mail.internet.MimeMessage;
import jakarta.ws.rs.core.Response;
import org.keycloak.testframework.annotations.InjectAdminClient;
@ -45,11 +48,13 @@ import org.keycloak.testframework.realm.RealmConfigBuilder;
import org.keycloak.testframework.server.KeycloakUrls;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.fail;
import static org.keycloak.representations.idm.ComponentRepresentation.SECRET_VALUE;
@ -115,7 +120,7 @@ public class SMTPConnectionTest {
RealmRepresentation realmRep = realm.toRepresentation();
realmRep.setSmtpServer(smtpMap("127.0.0.1", "3025", "auto@keycloak.org", "true", null, null,
"admin@localhost", password, null, null));
"admin@localhost", password, null, null, null));
managedRealm.updateWithCleanup(r -> r.update(realmRep));
mailServer.credentials("admin@localhost", password);
@ -220,9 +225,54 @@ public class SMTPConnectionTest {
}
@Test
@Order(9)
public void testAllowUTF8() throws Exception {
// utf8 on from not allowed if allowutf8 not enabled
Response response = adminClient.realms().realm(managedRealm.getName()).testSMTPConnection(settings("127.0.0.1", "3025", "autoñ@keycloak.org",
null, null, null, null, null));
assertStatus(response, 500);
// utf-8 on from but in domain part is allowed and transformed
response = adminClient.realms().realm(managedRealm.getName()).testSMTPConnection(settings("127.0.0.1", "3025", "auto@keycloakñ.org",
null, null, null, null, null));
assertStatus(response, 204);
assertMailReceived("auto@xn--keycloak-k3a.org");
// utf8 on from allowed if allowutf8 enabled
response = adminClient.realms().realm(managedRealm.getName()).testSMTPConnection(smtpMap("127.0.0.1", "3025", "autoñ@keycloak.org",
null, null, null, null, null, null, null, "true"));
assertStatus(response, 204);
assertMailReceived();
// utf8 on address
AccessToken token = oAuthClient.parseToken(adminClient.tokenManager().getAccessToken().getToken(), AccessToken.class);
UserResource userRes = adminClient.realm("default").users().get(token.getSubject());
UserRepresentation userRep = userRes.toRepresentation();
final String previousEmail = userRep.getEmail();
userRep.setEmail("adminñ@localhost");
userRes.update(userRep);
try {
// not allowed on address if allowutf8 not enabled
response = adminClient.realms().realm(managedRealm.getName()).testSMTPConnection(settings("127.0.0.1", "3025", "auto@keycloak.org",
null, null, null, null, null));
assertStatus(response, 500);
// allowed on address if allowutf8 enabled
response = adminClient.realms().realm(managedRealm.getName()).testSMTPConnection(smtpMap("127.0.0.1", "3025", "auto@keycloak.org",
null, null, null, null, null, null, null, "true"));
assertStatus(response, 204);
assertMailReceived();
} finally {
userRep.setEmail(previousEmail);
userRes.update(userRep);
}
}
private Map<String, String> settings(String host, String port, String from, String auth, String ssl, String starttls,
String username, String password) throws Exception {
return smtpMap(host, port, from, auth, ssl, starttls, username, password, "", "");
return smtpMap(host, port, from, auth, ssl, starttls, username, password, "", "", null);
}
private Map<String, String> settings(String host, String port, String from, String auth, String ssl, String starttls,
@ -251,7 +301,7 @@ public class SMTPConnectionTest {
}
private Map<String, String> smtpMap(String host, String port, String from, String auth, String ssl, String starttls,
String username, String password, String replyTo, String envelopeFrom) {
String username, String password, String replyTo, String envelopeFrom, String allowutf8) {
Map<String, String> config = new HashMap<>();
config.put("host", host);
config.put("port", port);
@ -264,6 +314,9 @@ public class SMTPConnectionTest {
config.put("password", password);
config.put("replyTo", replyTo);
config.put("envelopeFrom", envelopeFrom);
if (allowutf8 != null) {
config.put("allowutf8", allowutf8);
}
return config;
}
@ -283,10 +336,13 @@ public class SMTPConnectionTest {
response.close();
}
private void assertMailReceived() {
private void assertMailReceived(String... from) {
if (mailServer.getReceivedMessages().length == 1) {
try {
MimeMessage message = mailServer.getReceivedMessages()[0];
if (from.length > 0) {
assertArrayEquals(from, Arrays.stream(message.getFrom()).map(Address::toString).toArray(String[]::new));
}
assertEquals("[KEYCLOAK] - SMTP test message", message.getSubject());
mailServer.runCleanup();
} catch (Exception e) {