Skip to content

refactor(mimefactory): separate rendering of message payload and sendable message#8345

Open
link2xt wants to merge 2 commits into
mainfrom
link2xt/last_added_location_timestamp
Open

refactor(mimefactory): separate rendering of message payload and sendable message#8345
link2xt wants to merge 2 commits into
mainfrom
link2xt/last_added_location_timestamp

Conversation

@link2xt

@link2xt link2xt commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

This change separates rendering into two separate steps:

  1. Rendering of the message payload without the From, Date and Autocrypt headers.
  2. Adding the From, Date and Autocrypt headers and possibly encrypting the message.

The goal is to have serializable result of the first step
that can be persisted in the database and sent later with any email address.
This way it will be possible to send queued messages over any relay.

This will make it possible not to remove all messages from the queue
when the sending relay is changed.
Currently changing configured_addr deletes everything from smtp table.

This change is however only a refactoring and does not implement any features.


This is a refactoring PR in preparation for automatic relay failover.

As a side effect it also makes possible to change the Date of the message for #8112 if we decide on this approach (unlikely).

Serializable mail is currently called mimefactory::QueuedMail. Everything except the public keys is trivially serializable, public keys should likely be serialized as recipient fingerprints rather than as OpenPGP certificates.

I also noticed that we likely can remove the concept of "hidden headers" which are headers that are sent on the mulipart/mixed level of unencrypted messages. They are used to send Chat-Edit headers and avatars in unencrypted messages. Sending avatars in unencrypted messages is not useful because they are not displayed anyway. And we can decide to make it impossible to edit and delete unencrypted messages. I have not changed anything in this PR, however, hidden headers work as before.

@link2xt link2xt changed the title fix: replace last_added_location_id with last_added_location_timestamp MimeFactory refactoring and fixes Jun 18, 2026
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch 4 times, most recently from 45102bc to 1956c42 Compare June 18, 2026 18:00
@link2xt link2xt marked this pull request as ready for review June 19, 2026 01:33
@link2xt link2xt marked this pull request as draft June 19, 2026 01:33
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch 3 times, most recently from 5f3c57c to f9c33de Compare June 20, 2026 01:02
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch from f9c33de to e314fc0 Compare June 20, 2026 04:25
@link2xt link2xt changed the base branch from main to link2xt/location-streaming-fixes June 20, 2026 04:26
@link2xt link2xt force-pushed the link2xt/location-streaming-fixes branch from 3d4610c to b1b0e3f Compare June 20, 2026 04:29
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch 2 times, most recently from ce43887 to 6abd90c Compare June 20, 2026 19:48
@link2xt link2xt changed the title MimeFactory refactoring and fixes MimeFactory refactoring Jun 20, 2026
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch from 6abd90c to e350dd6 Compare June 20, 2026 20:01
@link2xt link2xt changed the base branch from link2xt/location-streaming-fixes to main June 20, 2026 20:02
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch 3 times, most recently from a18ab11 to 2e20218 Compare June 20, 2026 20:38
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch 5 times, most recently from a39da96 to 1c19dd8 Compare June 22, 2026 10:07
@link2xt link2xt changed the base branch from main to link2xt/location-streaming-fixes June 22, 2026 10:08
@link2xt link2xt changed the title MimeFactory refactoring refactor(mimefactory): separate rendering of message payload and sendable message Jun 22, 2026
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch from 1c19dd8 to efe7d0f Compare June 22, 2026 10:10
@link2xt link2xt force-pushed the link2xt/location-streaming-fixes branch from 28522ff to b34bc43 Compare June 26, 2026 08:58
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch 2 times, most recently from 606d7c7 to 23c8c8a Compare June 26, 2026 18:15
@link2xt link2xt force-pushed the link2xt/location-streaming-fixes branch from b34bc43 to c85e622 Compare June 26, 2026 18:15
@link2xt link2xt marked this pull request as ready for review June 26, 2026 18:24
@link2xt link2xt requested review from Hocuri, iequidoo and j-g00da June 27, 2026 14:34
Comment thread src/mimefactory.rs
Comment thread src/mimefactory.rs Outdated
Comment thread src/mimefactory.rs
/// Consumes a `MimeFactory` and renders it into a message which is then stored in
/// `smtp`-table to be used by the SMTP loop
#[expect(clippy::arithmetic_side_effects)]
pub(crate) async fn pre_render(

@iequidoo iequidoo Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe render_queued() or smth like this? pre_render() is somewhat "vague", and i'm afraid it may be confused with pre-messages.

EDIT: render_queued() sounds incorrect because it should mention the source object (MimeFactory), not the returned QueuedMail. Maybe into_queued_mail()?

@Hocuri Hocuri Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

into_queued_mail() sounds best to me. I agree that pre_render could be confused with pre-messages.

Comment thread src/mimefactory.rs
Comment thread src/mimefactory.rs
Comment thread src/mimefactory.rs
from_header.write_header(&mut inner_headers, 6)?;

if is_encrypted {
let unencrypted_from = Address::new_address(None::<&'static str>, from_addr.to_string());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let unencrypted_from = Address::new_address(None::<&'static str>, from_addr.to_string());
let unencrypted_from = Address::new_address(None::<&'static str>, from_addr.clone());

Comment thread src/mimefactory.rs

if should_attach_pubkey {
let aheader = Aheader {
addr: from_addr.clone(),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
addr: from_addr.clone(),
addr: from_addr,

Comment thread src/mimefactory.rs
verified: false,
};
let autocrypt_header = mail_builder::headers::raw::Raw::new(aheader.to_string());
if is_encrypted {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be

let headers = match is_encrypted {
    true => &mut inner_headers,
    false => &mut outer_headers,
};
...

to deduplicate the code a bit

Comment thread src/mimefactory.rs
};

// Encrypt to self unconditionally,
// even for a single-device setup.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"... to not reveal which setup we have", IIRC.

@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch from 23c8c8a to 9351ae3 Compare June 30, 2026 15:53
@link2xt link2xt force-pushed the link2xt/location-streaming-fixes branch from c85e622 to 908b761 Compare June 30, 2026 15:53
Base automatically changed from link2xt/location-streaming-fixes to main July 1, 2026 12:02
link2xt added 2 commits July 1, 2026 12:09
…able message

This change separates rendering into two separate steps:
1. Rendering of the message payload without the From, Date and Autocrypt headers.
2. Adding the From, Date and Autocrypt headers and possibly encrypting the message.

The goal is to have serializable result of the first step
that can be persisted in the database and sent later with any email address.
This way it will be possible to send queued messages over any relay.

This will make it possible not to remove all messages from the queue
when the sending relay is changed.
Currently changing `configured_addr` deletes everything from `smtp` table.

This change is however only a refactoring and does not implement any features.
@link2xt link2xt force-pushed the link2xt/last_added_location_timestamp branch from 9351ae3 to 598ac45 Compare July 1, 2026 12:10
Comment thread src/mimefactory.rs
} else {
SeipdVersion::V1
};
let display_name = if is_securejoin_message && !is_encrypted {

@iequidoo iequidoo Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, this is apparently not the correct implementation of #7396 , see the protocol flow chart -- Alice should attach her profile (avatar, displayname, etc.) only at the final step, but not in the "vc-pubkey" message which is encrypted with AUTH. It seems this doesn't create any security issue, but it's just enough to send the profile at the final step. Also if we let Alice review "Securejoin requests" in the future before fully executing them, showing Bob's profile to Alice first (not otherwise) makes sense:

Since we want to give Alice the opportunity to review join requests (as a future improvement that doesn't require breaking the protocol), Alice can't send any private data in the second message. [...]

EDIT: For Bob it's even worse because he sends his displayname already in "vc-request-pubkey" apparently, so e.g. other broadcast subscribers can see it.

CC @Hocuri

This is not related to the refactoring, but i'd like this to be reviewed by others.

@iequidoo iequidoo Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this misunderstanding comes from improper naming of is_encrypted which is evaluated by this function:

    pub fn will_be_encrypted(&self) -> bool {
        self.encryption_pubkeys.is_some()
    }

Could it be renamed in a separate commit to e.g. is_pubkey_encrypted? I've created #8399 and tested it with this PR as well, everything works as expected actually.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, the naming of will_be_encrypted is correct. If the message is symmetrically-encrypted, then encryption_pubkeys is Some(vec![]). As @link2xt said elsewhere, would be a nice refactoring to put this into an enum instead in order to prevent such confusion.

The "vc-pubkey" and "vc-request-pubkey" messages don't use the function here. Instead, they are created by render_symm_encrypted_securejoin_message. This is why the code works as correctly. Thanks for writing a test for it!

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making it clear! I think that just renaming encryption_pubkeys to encryption would make the code more understandable (because it's actually not only about pubkeys). And as for this check, it could use self.encryption.is_none_or(|pubkeys| pubkeys.is_empty()), but maybe it's better to add an assertion that we're not rendering a symmetrically encrypted SecureJoin message here.

Comment thread src/mimefactory.rs
/// - `Message-ID`
///
/// Encrypts and signs the message if necessary.
pub(crate) fn render_queued_mail(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be a member function of QueuedMail, even if there are some extra params

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems equally fine as a standalone function

Comment thread src/mimefactory.rs
.await?;
let side_effects = RenderSideEffects {
subject: "Secure-Join".to_string(),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

@Hocuri

Hocuri commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

I also noticed that we likely can remove the concept of "hidden headers" which are headers that are sent on the mulipart/mixed level of unencrypted messages. They are used to send Chat-Edit headers and avatars in unencrypted messages. Sending avatars in unencrypted messages is not useful because they are not displayed anyway. And we can decide to make it impossible to edit and delete unencrypted messages. I have not changed anything in this PR, however, hidden headers work as before.

Agreed, there is no need to support chat-like features in unencrypted messages anymore. Hidden headers are from a time when we wanted to enable nice DC-to-DC communication via unencrypted messages with avatars etc.

@r10s

r10s commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

ftr, some history: we had late-mime-generation until 2018 and switched to early-mime-generation for various reasons, mainly speed and disappearing-messages, thid old core-c issue shows more detailes deltachat/deltachat-core#427

while performance might still be a concern, esp when there are only short time windows to get things out, iiuc, in this PR most data to be sent out are still generated early, so eg it is fine if database records the message originates from are deleted, this is important eg. for disappearing messages

that time, however, mime-generation included encryption, so doing that late uses the state of the database when message goes out, not the state when send button is pressed, which led to all kind of issues.

if i get it corectly, this is not the case with this PR, the mime

@Hocuri

Hocuri commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

I made some measurements on my 5 year old middle-class phone (Fairphone 3), compiled in release mode.

FTR, this PR here doesn't switch to late encryption yet (it's just refactoring that will enable us to switch to late encryption). And yes, the plan is to switch to late encryption, but keep early mime generation.

Once we switch to late encryption, the function render_queued_mail will be executed before every sending attempt rather than just once. For an image, it took 0.1s - 0.2s, which seems fine. For a 10MB document, it took 3.5s, which is a lot, but also, I'm not sure if there is a realistic alternative, and most of the time users don't send 10MB documents. An advantage of late encryption will probably be that the message already appears in the chat before it's encrypted (we can't test that yet because late encryption is not implemented yet).

Text:
07-10 14:12:03.833 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2843: pre_render took 5.23625ms
07-10 14:12:03.846 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2853: render_queued_mail took 12.344167ms

Image:
07-10 14:12:31.241 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2787: post-message pre_render took 10.24974ms
07-10 14:12:31.442 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2802: post-message render_queued_mail took 202.123281ms
07-10 14:12:31.444 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2814: pre-message pre_render took 1.15401ms
07-10 14:12:31.446 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2824: pre-message render_queued_mail took 2.330677ms

Large image that was recoded: (note that the recoding itself happens before pre_render)
07-10 14:13:01.141 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2787: post-message pre_render took 7.337344ms
07-10 14:13:01.253 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2802: post-message render_queued_mail took 111.358542ms
07-10 14:13:01.254 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2814: pre-message pre_render took 1.570625ms
07-10 14:13:01.257 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2824: pre-message render_queued_mail took 3.268594ms

10MB document:
07-10 14:13:30.648 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2787: post-message pre_render took 162.277239ms
07-10 14:13:34.175 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2802: post-message render_queued_mail took 3.527979478s
07-10 14:13:34.177 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2814: pre-message pre_render took 1.408177ms
07-10 14:13:34.179 15848 15869 I DeltaChat: [accId=1] src/chat.rs:2824: pre-message render_queued_mail took 2.378177ms
Click to see the diff I used for measuring
diff --git a/src/chat.rs b/src/chat.rs
index 0ad2fd49d..9ee6bfc65 100644
--- a/src/chat.rs
+++ b/src/chat.rs
@@ -6,7 +6,7 @@
 use std::io::Cursor;
 use std::marker::Sync;
 use std::path::{Path, PathBuf};
-use std::time::Duration;
+use std::time::{Duration, Instant};
 
 use anyhow::{Context as _, Result, anyhow, bail, ensure};
 use chrono::TimeZone;
@@ -2780,10 +2780,17 @@ async fn render_mime_message_and_pre_message(
 
         let mut mimefactory_post_msg = mimefactory.clone();
         mimefactory_post_msg.set_as_post_message();
+        let start = Instant::now();
         let (queued_msg, side_effects) = Box::pin(mimefactory_post_msg.pre_render(context))
             .await
             .context("Failed to render post-message")?;
+        info!(
+            context,
+            "post-message pre_render took {:?}",
+            start.elapsed()
+        );
 
+        let start = Instant::now();
         let rendered_msg = mimefactory::render_queued_mail(
             queued_msg,
             &public_key,
@@ -2792,12 +2799,20 @@ async fn render_mime_message_and_pre_message(
             timestamp,
             side_effects,
         )?;
+        info!(
+            context,
+            "post-message render_queued_mail took {:?}",
+            start.elapsed()
+        );
 
         let mut mimefactory_pre_msg = mimefactory;
         mimefactory_pre_msg.set_as_pre_message_for(&rendered_msg);
+        let start = Instant::now();
         let (queued_pre_msg, pre_side_effects) = Box::pin(mimefactory_pre_msg.pre_render(context))
             .await
             .context("pre-message failed to render")?;
+        info!(context, "pre-message pre_render took {:?}", start.elapsed());
+        let start = Instant::now();
         let rendered_pre_msg = mimefactory::render_queued_mail(
             queued_pre_msg,
             &public_key,
@@ -2806,6 +2821,11 @@ async fn render_mime_message_and_pre_message(
             timestamp,
             pre_side_effects,
         )?;
+        info!(
+            context,
+            "pre-message render_queued_mail took {:?}",
+            start.elapsed()
+        );
 
         if rendered_pre_msg.message.len() > PRE_MSG_SIZE_WARNING_THRESHOLD {
             warn!(
@@ -2818,7 +2838,10 @@ async fn render_mime_message_and_pre_message(
 
         Ok((Some(rendered_pre_msg), rendered_msg))
     } else {
+        let start = Instant::now();
         let (queued_msg, side_effects) = Box::pin(mimefactory.pre_render(context)).await?;
+        info!(context, "pre_render took {:?}", start.elapsed());
+        let start = Instant::now();
         let rendered_msg = mimefactory::render_queued_mail(
             queued_msg,
             &public_key,
@@ -2827,6 +2850,7 @@ async fn render_mime_message_and_pre_message(
             timestamp,
             side_effects,
         )?;
+        info!(context, "render_queued_mail took {:?}", start.elapsed());
 
         Ok((None, rendered_msg))
     }

@Hocuri Hocuri left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review

Comment thread src/mimefactory.rs
Comment on lines +1258 to +1266
let (queued_mail, side_effects) = Box::pin(self.pre_render(context)).await?;
let rendered_mail = render_queued_mail(
queued_mail,
&public_key,
&secret_key,
from_addr,
timestamp,
side_effects,
)?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

render_queued_mail() is a blocking function (possibly blocking for multiple seconds) that is called without spawn_blocking() or block_in_place() both here and in multiple other places. This needs to be fixed in order not to block the executor

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, spawn_blocking() was removed from pk_encrypt(). I think, to protect from blocking the executor in such cases, we should use block_in_place() at the lower level, i.e. in pk_encrypt() while still calling spawn_blocking() here and in other high-level functions. So, even if spawn_blocking() is forgotten, the worst thing is other futures in the same task getting blocked, not other tasks. Nested block_in_place() should be fine (no-op), see discussion in tokio-rs/tokio#2327

Comment thread src/mimefactory.rs
/// Unencrypted queued message.
///
/// This message has both the headers and the body,
/// but without the From, Autocrypt and Date headers.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without the [...] Date header

Didn't we say that the Date header should continue to contain the time of the user pressing "Send", and if we add the time of sending the message out then it should go into a new header?

Comment thread src/mimefactory.rs
/// - `Message-ID`
///
/// Encrypts and signs the message if necessary.
pub(crate) fn render_queued_mail(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems equally fine as a standalone function

Comment thread src/mimefactory.rs
Comment on lines +272 to +284
from_header.write_header(&mut inner_headers, 6)?;

if is_encrypted {
let unencrypted_from = Address::new_address(None::<&'static str>, from_addr.to_string());
outer_headers.extend_from_slice(b"From: ");
unencrypted_from.write_header(&mut outer_headers, 6)?;

inner_headers.extend_from_slice(b"HP-Outer: From: ");
unencrypted_from.write_header(&mut inner_headers, 16)?;
} else {
outer_headers.extend_from_slice(b"From: ");
from_header.write_header(&mut outer_headers, 6)?;
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are a lot of magic numbers (6, 6, 16, 6, ...). Would be nice to extract a function like:

    fn add_header(
        name: &str,
        value: impl mail_builder::headers::Header,
        headers: &mut Vec<u8>,
    ) -> Result<()> {
        headers.extend_from_slice(name.as_bytes());
        value.write_header(headers, name.len())?;
        Ok(())
    }

Comment thread src/mimefactory.rs
Comment on lines +289 to +291
inner_headers.extend_from_slice(b"Date: ");
inner_headers.extend_from_slice(date.as_bytes());
inner_headers.extend_from_slice(b"\r\n");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for using extend_from_slice() rather than extend() everywhere? From what I understand, when both of them compile then they do the same except that extend() can be slightly faster? (because it uses copy rather than clone). And while performance doesn't matter, extend() would read more nicely than extend_from_slice() because it's shorter

Comment thread src/mimefactory.rs
.unwrap()
.to_rfc2822();
outer_headers.extend_from_slice(b"Date: ");
outer_headers.extend_from_slice(unprotected_date.as_bytes());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm mildly concerned about building everything ourselves here rather than using mail-builder, because we may forget to escape some special characters. But probably it's fine.

Comment thread src/mimefactory.rs

if is_encrypted {
// Copy not protected headers to outer headers.
let (parsed_headers, _index) = mailparse::parse_headers(&raw_message)?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR, we're first building the message and then parsing it again, but this parse_headers call takes just 5µs-80µs in my measurements (5 year old middle-class phone, release mode), and code-wise it seems like the easiest solution, so, it's good as-is.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...although the only header we actually need is Chat-Is-Post-Message. For all the others (Subject, To, Chat-Version) it's already clear that they should be added and with which value. So, the code here could be slightly simplified by putting pre_message_mode on QueuedMail, and then always mechanically adding these four outer headers. Then we wouldn't need to parse the message here and iterate over all headers.

But it's fine as-is, too

Comment thread src/mimefactory.rs
} else if header_name == "subject" {
&b"[...]"[..]
} else if header_name == "to" {
&b"To: \"hidden-recipients\""[..]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will lead to a header that looks like To: To: "hidden-recipients". IIRC it needs to be To: "hidden-recipients": ;, i.e. a group of recipients with 0 entries. I just tested it, GMX rejects mails sent with this PR because of a malformed To header.

Apparently we have no tests at all for how a rendered outer message should look like?

Comment thread src/mimefactory.rs
Comment on lines +409 to +410
let mut full_raw_message = inner_headers.clone();
full_raw_message.append(&mut raw_message);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This leaves raw_message empty but still accessible (not dropped), so that someone might accidentally use it. Using extend would consume it instead, which seems nicer. Also, there is no need to clone inner_headers since it's equally not needed afterwards.

Suggested change
let mut full_raw_message = inner_headers.clone();
full_raw_message.append(&mut raw_message);
let mut full_raw_message = inner_headers;
full_raw_message.extend(raw_message);

Comment thread src/mimefactory.rs
};

let mut full_message = outer_headers;
full_message.append(&mut message);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here:

Suggested change
full_message.append(&mut message);
full_message.extend(message);

Comment thread src/mimefactory.rs
// <https://www.rfc-editor.org/rfc/rfc9787.html#structural-header-fields>.
continue;
}
let header_value = if header_name == "mime-version"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

header_name can't be mime-version at this place. We checked just a few lines above, and continueed if so.

I guess it can just eb removed from here.

Comment thread src/mimefactory.rs

if is_encrypted {
// Copy not protected headers to outer headers.
let (parsed_headers, _index) = mailparse::parse_headers(&raw_message)?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...although the only header we actually need is Chat-Is-Post-Message. For all the others (Subject, To, Chat-Version) it's already clear that they should be added and with which value. So, the code here could be slightly simplified by putting pre_message_mode on QueuedMail, and then always mechanically adding these four outer headers. Then we wouldn't need to parse the message here and iterate over all headers.

But it's fine as-is, too

Comment thread src/mimefactory.rs
Comment on lines +1250 to +1252
/// Helper function render the messages that are not queued.
///
/// Used for MDNs because their payload is rendered right before sending.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Helper function render the messages that are not queued.
///
/// Used for MDNs because their payload is rendered right before sending.
/// Helper function that renders a message without queuing it first.
///
/// Used for MDNs because they are fully rendered and sent in one go,
/// rather than first creating a [`QueuedMail`] and sending it later.

Comment thread src/mimefactory.rs
} else {
SeipdVersion::V1
};
let display_name = if is_securejoin_message && !is_encrypted {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, the naming of will_be_encrypted is correct. If the message is symmetrically-encrypted, then encryption_pubkeys is Some(vec![]). As @link2xt said elsewhere, would be a nice refactoring to put this into an enum instead in order to prevent such confusion.

The "vc-pubkey" and "vc-request-pubkey" messages don't use the function here. Instead, they are created by render_symm_encrypted_securejoin_message. This is why the code works as correctly. Thanks for writing a test for it!

Comment thread src/mimefactory.rs
Comment on lines +2525 to 2530
let side_effects = RenderSideEffects {
subject: "Secure-Join".to_string(),

wrap_encrypted_part(encrypted)
..Default::default()
};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to care about the subject of securejoin messages anymore?

Suggested change
let side_effects = RenderSideEffects {
subject: "Secure-Join".to_string(),
wrap_encrypted_part(encrypted)
..Default::default()
};
let side_effects = RenderSideEffects::default();

Comment thread src/mimefactory.rs
@@ -2022,84 +2293,26 @@ struct HeadersByConfidentiality {
/// See [`HeadersByConfidentiality`] for more info.
fn group_headers_by_confidentiality(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put a comment here that this function isn't really needed anymore and can be removed once we remove hidden headers?

Comment thread src/mimefactory.rs

Ok(message)
/// Renders MIME part into a vector.
pub(crate) fn part_to_vec(message: MimePart<'static>) -> Vec<u8> {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rename this function to part_to_bytes? To better describe what this function does without looking at the function signature (a "vec" could also be a vec of lines, or a vec of headers, or whatever)

Comment thread src/mimefactory.rs
Comment on lines +289 to +290
inner_headers.extend_from_slice(b"Date: ");
inner_headers.extend_from_slice(date.as_bytes());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be easier to already add the Date in pre_render()? Then the timestamp wouldn't need to be passed through. For the randomized outer date, the current time can be used rather than the time of clicking "Send".

Comment thread src/mimefactory.rs
Comment on lines +1504 to +1507
let message = message.header(
"Message-ID",
mail_builder::headers::message_id::MessageId::new(rfc724_mid.clone()),
);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed here? Does it add the Message-Id header to the inner part in order to reduce problems with MUAs messing with it?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just in case, currently we also put Message-Id into hidden headers, so this PR doesn't change this, and i think this is indeed to protect from servers messing with it

@iequidoo iequidoo self-requested a review July 12, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants