How to write .msg file

Sep 20, 2011 at 3:23 PM

I just downloaded your pst port and it is working great. Any idea how to write out an .msg file given an EntryId? If it is not currently supported I will gladly take a crack at it if you can provide some guidance.

Thanks in advance.

Developer
Sep 21, 2011 at 8:25 PM

A .msg file is a complete different type of beast than a PST.  The file format is here: http://msdn.microsoft.com/en-us/library/cc463912.aspx

It would do well to be a separate project from pstsdk.net, simply because it's such a large undertaking and is related more to Outlook than Psts.  The code here: http://www.codeproject.com/KB/office/reading_an_outlook_msg.aspx works really well, uses PInvoke to get read a pst.  I don't know about writing.  It may be a good place to start looking, however.

Sep 22, 2011 at 9:34 PM

Thanks for the reply.

I solved my problem by using your MimeMessageWriter to output an .eml file and then I used Redemption to convert the .eml to a .msg.

Nov 17, 2011 at 10:12 PM

After some work I was able to write an msg file from a Message object with some caveats. The main caveat is embedded messages don't work quite right and I get a weird error from Outlook when I click on a Recipients display name. Other than that simple messages with attachments and embedded objects seems to work.

Thanks a lot for this project I have found it quite useful.

		public class StructuredStorageMessageWriter : IMessageWriter
		{
			/**
			 * Converts a two byte array to an integer
			 * @param b a byte array of length 2
			 * @return an ulong
			 */
			private static ulong ByteArrayToUInt64(byte [] arr) 
			{
		    ulong ul = 0;

				for (int i = arr.Length - 1; i >= 0; i--)
				{
					ul <<= 8;
					ul |= (ulong)(byte)(arr[i] & 0xFF);
				}

				return ul;
			}

			#region private strings - storage and stream
			private const string nameOfNameIDStorage = "__nameid_version1.0";
			private const string nameOfSubStorageStream_Format = "__substg1.0_{0}{1}";
			private const string nameOfRecipStorage_Format = "__recip_version1.0_#{0}";
			private const string nameOfAttachStorage_Format = "__attach_version1.0_#{0}";
			private const string nameOfPropertiesStream = "__properties_version1.0";
			private const string nameOfEmbeddedMessageStream = "__substg1.0_3701000D";
			#endregion

			#region private methods - storage and stream names
			private string MakeSubStorageStreamName(int propID, int propType)
			{
				return string.Format(nameOfSubStorageStream_Format, ((int)propID).ToString("X4"), ((int)propType).ToString("X4"));
			}

			private string MakeRecipStorageName(int recipientIncrement)
			{
				return string.Format(nameOfRecipStorage_Format, recipientIncrement.ToString().PadLeft(8, '0'));
			}

			private string MakeAttachStorageName(int attachmentIncrement)
			{
				return string.Format(nameOfAttachStorage_Format, attachmentIncrement.ToString().PadLeft(8, '0'));
			}
			private void WritePropertyData(byte[] propdata, PropId propid, PropertyType proptype, uint readtype, OpenMcdf.CFStorage cfstorage, OpenMcdf.CFStream cfstream)
			{
				cfstream.AppendData(BitConverter.GetBytes((short)proptype.Value));
				cfstream.AppendData(BitConverter.GetBytes((short)propid.Value));
				cfstream.AppendData(BitConverter.GetBytes(readtype));
				if (PropertyUtils.IsVariableLength(proptype))
				{
					// if property id is Subject needed to chop off first four bytes
					if (propid.Value == 0x0037)
						propdata = propdata.Skip(4).ToArray();

					if (proptype == PropertyType.KnownValue.prop_type_wstring && propdata.Length > 0)
						propdata = propdata.Concat(new byte[] { 0x00, 0x00 }).ToArray();
					cfstream.AppendData(BitConverter.GetBytes(Convert.ToUInt64(propdata.Length))); // + ((PropertyType)propid.PropertyType == PropertyType.KnownValue.prop_type_wstring ? 2 : 0))));
					cfstorage.AddStream(MakeSubStorageStreamName(propid, proptype)).SetData(propdata);
				}
				else
					cfstream.AppendData(BitConverter.GetBytes(ByteArrayToUInt64(propdata)));
			}

			#endregion

			#region IMessageWriter Members

			public void Write(IMessage message, Stream stream)
			{
				// the minimum storage configuration for an msg is the following
				// Root
				//   __nameid_version1.0
				//     __substg1.0_00020102
				//     __substg1.0_00030102
				//     __substg1.0_00040102
				//  __properties_version1.0
				// this is the Root
				OpenMcdf.CompoundFile cf = new OpenMcdf.CompoundFile();

				Write(message, stream, cf.RootStorage);

				cf.Save(stream);
				cf.Close();
			}

			private void Write(IMessage message, Stream stream, OpenMcdf.CFStorage rootstorage)
			{
				// the minimum storage configuration for an msg is the following
				// Root
				//   __nameid_version1.0
				//     __substg1.0_00020102
				//     __substg1.0_00030102
				//     __substg1.0_00040102
				//  __properties_version1.0
				// this is the Root
//				OpenMcdf.CompoundFile cf = new OpenMcdf.CompoundFile();
				// this storage is __nameid_version1.0
				OpenMcdf.CFStorage cfstorage = rootstorage.AddStorage(nameOfNameIDStorage);
				// these streams are __substg1.0_000X0102
				for (int i = 2; i <= 4; i++)
					cfstorage.AddStream(MakeSubStorageStreamName(i, 0x102));
				// this stream is __properties_version1.0
				OpenMcdf.CFStream cfstream = rootstorage.AddStream(nameOfPropertiesStream);

				// we need to add header to the stream __properties_version1.0
				// 0x0 + message.RecipientCount + message.AttachmentCount + message.RecipientCount + message.AttachmentCount + 0x0
				// all values in 32-bit unsigned int
				cfstream.SetData(new byte[8] { 0, 0, 0, 0, 0, 0, 0, 0 });
				cfstream.AppendData(BitConverter.GetBytes(message.RecipientCount));
				cfstream.AppendData(BitConverter.GetBytes(message.AttachmentCount));
				cfstream.AppendData(BitConverter.GetBytes(message.RecipientCount));
				cfstream.AppendData(BitConverter.GetBytes(message.AttachmentCount));
				cfstream.AppendData(new byte[8] { 0, 0, 0, 0, 0, 0, 0, 0 });

				#region write out message properties
				// the message property stream has to have a property id of 0x340D for the msg to be parsed by Outlook
				// there are other property ids listed below in the switch statement that are added by Outlook when an email
				// is saved as an msg file but I do not think they are required
				foreach (PropId propid in message.Properties.Where(x => x.Value != 0x340D).Concat(new List<PropId>() { 0x340D }).OrderBy(x => x))
				{
					uint readtype = 0x02;
					PropertyType proptype = PropertyType.KnownValue.prop_type_null;
					byte[] propdata = null;

					if (message.PropertyExists(propid))
					{
						proptype = message.GetPropertyType(propid);
						propdata = message.ReadProperty(propid);
					}
					else
					{
						// this switch statement provides default values for the property id
						// if it is not fond in the message's property bag
						switch (propid)
						{
							case 0x0FF4:
								propdata = new byte[] { 2, 0, 0, 0, 0, 0, 0, 0 };
								proptype = 0x03;
								break;
							case 0x0FF7:
								propdata = new byte[] { 0, 0, 0, 0, 0, 0, 0, 0 };
								proptype = 0x03;
								break;
							case 0x3007:
								propdata = new byte[] { 0, 0, 0, 0, 0, 0, 0, 0 };
								proptype = 0x40;
								break;
							case 0x3008:
								propdata = new byte[] { 0, 0, 0, 0, 0, 0, 0, 0 };
								proptype = 0x40;
								break;
							case 0x0E04:
								propdata = new byte[] { };
								proptype = 0x1F;
								break;
							case 0x0E03:
								propdata = new byte[] { };
								proptype = 0x1F;
								break;
							case 0x0E02:
								propdata = new byte[] { };
								proptype = 0x1F;
								break;
							case 0x340D:
								propdata = new byte[] { 0x79, 0x0E, 0x04, 0, 0, 0, 0, 0 };
								proptype = 0x03;
								break;
						}
					}
					WritePropertyData(propdata, propid, proptype, readtype, rootstorage, cfstream);
				}
				#endregion

				#region write out recipients
				int recipInc = 0;
				foreach (IRecipient recip in message.Recipients)
				{
					// add storage for stream __recip_version1.0_#
					OpenMcdf.CFStorage recipstorage = rootstorage.AddStorage(MakeRecipStorageName(recipInc++));
					// add property stream __properties_version1.0 to storage __recip_version1.0_#
					OpenMcdf.CFStream recipstream = recipstorage.AddStream(nameOfPropertiesStream);
					// add header to the recipient stream __properties_version1.0 
					// 0x00000000
					recipstream.AppendData(new byte[] { 0, 0, 0, 0, 0, 0, 0, 0 });
					// write out properties
					foreach (PropId propid in recip.Properties.OrderBy(x => x.Value))
						WritePropertyData(recip.ReadProperty(propid), propid, recip.GetPropertyType(propid), (uint)0x02, recipstorage, recipstream);
				}
				#endregion

				#region write out attachments
				int attachInc = 0;
				foreach (IAttachment attach in message.Attachments) //.Where(x => !x.IsMessage))
				{
					// add storage for stream __attach_version1.0_#
					OpenMcdf.CFStorage attachstorage = rootstorage.AddStorage(MakeAttachStorageName(attachInc));
					// add property stream __properties_version1.0 to storage __attach_version1.0_#
					OpenMcdf.CFStream attachstream = attachstorage.AddStream(nameOfPropertiesStream);
					// add header to the attachment stream __properties_version1.0 
					// 0x00000000
					attachstream.AppendData(new byte[] { 0, 0, 0, 0, 0, 0, 0, 0 });
					// write out properties - 0x0E21 has to be in the property list
					// I added the Where and Concat clauses to make sure it is
					foreach (PropId propid in attach.Properties.Where(x => x.Value != 0x0E21).Concat(new List<PropId>() { 0x0E21 }).OrderBy(x => x))
					{
						uint readtype = 0x02;
						PropertyType proptype = PropertyType.KnownValue.prop_type_null;
						byte[] propdata = null;
						if (attach.PropertyExists(propid))
						{
							proptype = attach.GetPropertyType(propid);
							propdata = attach.ReadProperty(propid);
						}
						else
						{
							switch (propid)
							{
								case 0x0E21:
									proptype = 0x03;
									propdata = BitConverter.GetBytes(Convert.ToInt64(attachInc));
									break;
							}
						}
						WritePropertyData(propdata, propid, proptype, readtype, attachstorage, attachstream);
					}
					if (attach.IsMessage)
						Write(attach.OpenAsMessage(), stream, attachstorage.AddStorage(nameOfEmbeddedMessageStream));
					attachInc++;
				}
				#endregion
			}
			#endregion
		}

Apr 20, 2012 at 12:41 PM

Can I use CompoundFile to duplicate a Outlook msg, modify his properties and create new one?.

Thx.

Jun 14, 2012 at 6:03 PM

Thank you for the above sample to write an msg file from a Message object. I am facing a problem with the MSG files so created, these do not open when we double click in file system as a Outlook Item. Gives error "Cannot read the item.

Kindly suggest solution. Thank you.

Developer
Jun 14, 2012 at 8:49 PM

Hi softpal,

It's hard to say without looking at your code or knowing what kind of message object you're trying to create.  The above solution that kmrcodeplex attached is not a complete sample, there are a few key things missing, as they've even admitted themselves.  It doesn't support embedded or custom attachments properly.  It doesn't actually populate the nameid storage.  It also hard codes the value for PidTagStoreSupportMask, with the following flags:

UNICODE_OK
STORE_RTF_OK
STORE_READONLY
STORE_OLE_OK
STORE_MV_PROPS_OK
STORE_MODIFY_OK
STORE_ENTRYID_UNIQUE
STORE_CREATE_OK
STORE_CATEGORIZE_OK
STORE_ATTACH_OK

STORE_MV_PROPS_OK <- that might be an issue if your message has Multi-Valued properties, because they're stored *completely* different in an MSG than they are in the PST.  It also doesn't check for maximum limits for attachments and recipient objects, which might(?) be a lower limit than in a PST.

Also, with the UNICODE_OK flag set, there can't be ANY ansi strings present in the MSG.  Those should be converted to unicode.

The above posted solution is a good start, but doesn't cover all the bases.  There are probably other issues but those are the ones that I can see off the top of my head.  I would look at the MSG spec here (http://msdn.microsoft.com/en-us/library/cc463912(v=exchg.80)) and compare it with the output.  Also keep in mind that there are bugs in outlook related to having html bodies in MSG files.  Outlook will convert the HTML bodies to RTF instead of fixing the issue (I can't find the specific microsoft KB article).  I believe one of the issues is that if you try sending an MSG file you've opened that has an HTML body and no RTF body, the email will be sent without a body at all.

 

We don't have official MSG support in this project and I don't see it happening in the near future.  However, I hope that the above information can help you in some way.

 

Thanks,

Christopher

Jun 15, 2012 at 2:06 AM

Thank you for your quick response. 

Kindly have a look at http://www.codeproject.com/Articles/28209/Outlook-Drag-and-Drop-in-C and http://www.codeproject.com/Articles/32899/Reading-an-Outlook-MSG-File-in-C 

Please advice me if these can be in any way be integrated and utilized along with PSTSDK IMessage object to allow to have the functionality of Save the email as MSG.

Thank you.

Jun 15, 2012 at 2:57 AM

Hi softpal - The above code I posted was a first shot at trying to write out an msg and as Chris points out insufficient. My current version addresses some of Chris's concerns such as support for ANSI psts and named property mapping. I still do not support multi-valued properties. I am not sure what is meant by "check for maximum limits for attachments and recipient" - I am not aware there are any limits to the attachments and recipients. I can send you the code if you like.

Jun 15, 2012 at 5:13 AM

Thank you for your response, I am interested - please send me the latest updated code (info at softpal dot com - info@softpal.com). And also advice me how to move further on this and have a complete fully working sample for saving as MSG for everyone's benefit.

Thank you.

Developer
Jun 15, 2012 at 5:45 PM

@kmrcodeplex - There's a limit for attachments and recipients, 2048 per storage type.  I've never seen that many attachments, but I've seen the number of recipients hit that before.  Glad to hear you have more support for MSGs.  I might be able to help with the multi-valued properties.  I have some existing code that I can probably adapt into yours.  Just let me know.

Jun 15, 2012 at 7:26 PM
I have attached a zip file with all of my implementations for msg representations. The StructuredStorageMessageWriter code writes an instance of Message to a stream. I hope you find it useful and please send back any criticisms or suggestions.

Thanks

Kevin

On Fri, Jun 15, 2012 at 12:45 PM, ccurrens <notifications@codeplex.com> wrote:

From: ccurrens

@kmrcodeplex - There's a limit for attachments and recipients, 2048 per storage type. I've never seen that many attachments, but I've seen the number of recipients hit that before. Glad to hear you have more support for MSGs. I might be able to help with the multi-valued properties. I have some existing code that I can probably adapt into yours. Just let me know.

Read the full discussion online.

To add a post to this discussion, reply to this email (pstsdknet@discussions.codeplex.com)

To start a new discussion for this project, email pstsdknet@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Jun 16, 2012 at 12:53 AM
Edited Jun 17, 2012 at 3:08 PM

I did not receive the sample code ZIP file. Please send once again.

I am facing few issues in the current version of code.

1. in "\pstSDK\pstsdknet\trunk\pstsdk.net\layer\util\PropertyUtils.cs"  method "IsVariableLength" is missing. This method is referenced in the above sample code.

2. While writing a MSG, need to take care of Encoding.UTF8 and Encoding.Unicode for String property type.

3. After using the above sample, and saving a MSG in file system. I am facing a problem with the MSG files so created, these do not open when we double click in file system as a Outlook Item. Gives error "Cannot read the item.

Note: I have used the above sample code along with "http://sourceforge.net/projects/openmcdf/"

Thank you

Jun 19, 2012 at 3:32 AM

Kindly reply, I am eagerly waiting for the updated sample code.

Thank you

Jan 24, 2013 at 8:35 AM

kmrcodeplex - I would be delighted if you could send me a copy of your code as well, or post the files here or to github, as I think this is probably an issue a lot of people have, and a collaborative approach could lead to a complete solution

Many thanks

 

Trenton

trentondj (at) secretbear.com