I'm running into a problems when sending messages in Cyrillic with Contact form (as well as with Webform) on a Backdrop CMS site:

The message is received from the sender's email and the Cyrillic text reads normally when reading the message in Thunderbird / Windows 10 email client.

However, when receiving the message by an email client on an Android smartphone (the e-mail client is K9 Mail), the Cyrillic text is replaced with unintelligible characters.

Here is an example of the K9 email client for Android:

�������� Test (abc@email.com) �������������� ���������������� �� ������
������������������ ���� �������������� https://example.com/contact.

�������������� ���������� ���� �������� Test: 12345678

�������� 3 Test 1 PHP 8.3

When reading the same message through Roundcube web-based e-mail that is part of CPanel's web server applications, Cyrillic in messages is also unintelligible:

Тест Test (abc@email.com) изпрати съобщение чрез
страницата за контакти https://example.com/contact.

Телефонен номер на Тест Test: 12345678

Тест 3 Test 1 PHP 8.3

In both examples, it can be seen that the problem affects only the part of the text that is in Cyrillic, but the Latin is read normally. The problem affects both the Subject and the body of the message.

This problem only occurs when the PHP version of the web server is changed to PHP 8 (the problem occurs equally with PHP 8.0, PH 8.1, PHP 8.2).

If the web server is running PHP 7.4 there is no problem and Cyrillic messages sent through the message form are read normally.

For comparison, I also sent a message via a Webform module message form:

When running the web server with PHP 7.4, there are no problems again.

Changing the PHP version to 8.0 results in the following message sent via Webform

�������������� ��� 11
�������������� ���� ������������, �������������� 19 , 2023 - 5:39pm

���������� ���� ����������������������:

����������: Test
������, ��������������: Test
��������������: 12345
Email: abc@email.com
������������ ���� ��������������: ���������������� � ���� ��������������:

Test web server in this test is running PHP 8.0. If the web server in is running PHP 7.4 there are no problems with Cyrillic content in messages.

The output shows for me that it seems the problem is valid for both the contact form module and the messages sent through the webform module.

This problem is preventing me from upgrading the web servers of the wesites that I administer (they are all managed by cPanel) to PHP 8, and the lifetime of PHP 7 is decreasing alarmingly.

I would greatly appreciate any advice on how to try to solve this problem. Maybe there are some message code table settings that can be edited in the Backdrop settings.

Comments

Hi amilenkov. Can you check the dblog to see if there are any errors right after sending the messages? Also are you using any of the modules: SMTP? Mime Mail? Mail System?

Thanks for the reply, but I have to apologize, I don't know what dblog means.

There are no error descriptions in Recent log messages. I can access the server through cPanel and FTP as well as phpMyAdmin but I don't know exactly where to check if these are the required tools.

I am not usung modules: SMTP? Mime Mail? Mail System? Only core message system.

Oh, I understand admin/reports/dblog No there are no error messages

Типmail

Date Неделя, Ноември 19, 2023 - 5:03pm

User Anonymous (not verified)

Location https://domain.bg/contact

Referrer https://domain.bg/contact

СъобщениеТест Test (abc@email.com) sent an email regarding Контакти.

Severitynotice

Hostname xxx.xxx.xxx.xxx

Operations

OK, so you are sending email with core functionality (no contrib), and there are no errors in the database log. Hmmm....

I would open an issue in the main Backdrop issue queue. This looks like a bug related to PHP 8. 

Hello argiepiano,

Above you wrote:

I would open an issue in the main Backdrop issue queue. This looks like a bug related to PHP 8.

If you are open an issue, could you send me a link so I can follow the discussion.

Or if I have to open an issue, could you tell me where to do it?

Thanks argiepiano, I have a github account and will open an issue. I just need to think about how to present the problem and illustrate it with examples. I asked the question to avoid duplicate issues.

Thank you!

Perhaps the reason this hasn't been noticed is that there aren't many users of Backdrop CMS who need Cyrillic.

As far as I am informed, I am the only user of the system in Bulgaria. However, I am responsible for the maintenance and development of more than 65 sites, and now all of them are with Backdrop CMS :) (except for one with Drupal 6)

That's probably right - not many other users sending email in Cyrillic. I know very little about multi-language emails in general, so I'm afraid I can't help, but if you post an issue you may get replies from people more familiar with this.

The issues described in this topic were posted as an issue on Github at:

https://github.com/backdrop/backdrop-issues/issues/6305

The issue was investigated by various experts of the Backdrop community and they concluded that the Cyrillic issues when migrating from PHP 7 to PHP 8 are not rooted in Backdrop CMS, but are likely due to issues in the PHP 8 web server configuration or the specific email client K9 Mail.

Now I want to lay out some few additional facts that became known to me subsequently, as well as the solution to the problem that I was able to find, which may be useful for other users facing a similar problem - and it can be related not only with the Cyrillic alphabet, but also with other languages and alphabets containing symbols other than those in the standard Latin alphabet.

1. The problems with Cyrillic (when PHP 8 is used on web server side) with  the K9 Mail client for Android seem to be of no small importance to Backdrop users with Baxkdrop CMS contact forms given the news that Mozilla has acquired control of the K9 Mail app and is currently rebranding it as Thunderbird for Android - which is likely to significantly increase the number to users who rely on it for their e-mail on smartphones:

https://github.com/thunderbird/thunderbird-android/releases

2. Cyrillic messaging tests conducted by my hosting provider via a simple static PHP contact form (without any CMS) installed on the same web server did not detect web server configuration issues with Cyrillic when moving from PHP 7 to PHP 8. I was told that similar messaging issues containing Cyrillic and non-standard Latin letters (e.g. german umlauts) for some email clients are reported for Drupal 7, for example there:

https://www.drupal.org/project/contact_emails/issues/3077899

3. I was surprised to find that exactly the same problem exists with a brand new installation of Drupal 7 and moving the web server from PHP 7 to PHP 8. To save space I'm not describing them again, the problems are the same as described at the beginning of this theme in the forum. This seems to confirm the conclusion that the cause of the problem is not in Backdrop CMS by itself since the identical problem is also observed in Drupal 7. In the additional tests, I found something else that I hadn't noticed in the previous tests - when moving from PHP 7 to PHP 8 there is another strange problem. When the server works with PHP 7 and the received message is answered with Reply, the answer is sent to the email address filled in the contact form. After switching to PHP 8, the response is sent to the address from which the message was sent via the contact form - this address, which is entered in admin/config/system/site-information as the site's email address. Unfortunately, my limited knowledge does not allow a deeper understanding of the real reasons for this phenomenon.

4. Finally, I want to share that all these problems are solved by installing the SMTP Authentication Support module https://backdropcms.org/project/smtp

In my case, I used an smtp server, which is a standard part of the shared hosting service and is available through the settings for each email account created through cPanel. Only difficulty was getting the correct settings for my server configuration as they are not described in the module documentation and I found them by searching the Internet. For me, smtp access via an encrypted connection was via TLS and port 587.

After installing and properly configuring the SMTP Authentication Support module, all Cyrillic problems were removed and Cyrillic messages are received correctly both through the Contact Form, Webform and Rules. When using this module, the problem with the electronic address for replying to a message with Reply was also solved - the reply is sent to the address filled in the contact form.

Finally, consulting this decision with my hosting provider, the experts there advised me to stick with this configuration because using an smtp server reduces the likelihood that various email service providers will rate messages sent through the contact form as spam.

Thanks, amelinkov! This is super helpful. It may also help developers debug core to see what's going on.