Think of all of the information your SD cards have seen, questionable pictures, sensitive data. Hardware hacker Bunnie Huang (link) gave an interesting presentation to the Chaos Computer Club Congress that highlighted the fact that SD cards are micro controllers with lax security that can easily be used to trick or hack you.

He first sets the tone of the presentation with this statement:

"lash memory is really cheap. So cheap, in fact, that it’s too good to be true. In reality, all flash memory is riddled with defects — without exception. The illusion of a contiguous, reliable storage media is crafted through sophisticated error correction and bad block management functions. This is the result of a constant arms race between the engineers and mother nature; with every fabrication process shrink, memory becomes cheaper but more unreliable. Likewise, with every generation, the engineers come up with more sophisticated and complicated algorithms to compensate for mother nature’s propensity for entropy and randomness at the atomic scale."

He then explains that engineers add smarts to counteract this act of god (which is where the power for evil comes in):

"These algorithms are too complicated and too device-specific to be run at the application or OS level, and so it turns out that every flash memory disk ships with a reasonably powerful micro-controller to run a custom set of disk abstraction algorithms. Even the diminutive microSD card contains not one, but at least two chips — a controller, and at least one flash chip (high density cards will stack multiple flash die)."

So these microprocessors contain special logic (algorithms) that detect defects and then only make available bits that are expected to be good. Also cards contain more space than shown as available (to ensure the promised amount stays available). This means that a "bad actor" can change the firmware on an SD card to copy data to this hidden storage space. This also means a card can show 16/32/64gB as available when it only contains 2GB (think of low cost no name SD cards from questionable Asian sources).

With this presentation, expect someone to develop an SD card hack to turn these little cheap trinkets into Arduino competitors. 

Not sure how we will secure our SD cards from compromise but I guess you should be buying your cards from reputable resellers and only buy top name brands.