After upgrading to Mac OS X 10.11 over Yosemite, I am likely have no trouble with the Mail app, but the mailbox size seems to be increased too much. It takes roughly 32 Gb instead of around 18 Gb that I have just seen in Yosemite before upgrading. What is going on?
On the same boat. I found out that the cause is duplication. The duplicates did not show in Mail but mailbox folders in Library had several versions of each message and attachment in subfolder structures.
To get rid of this trouble, you should delete the entire Mail folder in Library, then re-load all mail again.