Geek pop quiz: Brainstorming edition.

[theweaselking]

Imagine, for a moment, that you have a process (as in procedure, not as in "running program").

This process involves downloading an archive file, decrypting it, unpacking it, converting the files inside from one format to another, and printing them. After printing, the resulting prints must be proofread and checked for errors, and if errors are found, the file must be reprinted (possibly reprocessed then reprinted - depends on the kind of error).

Added difficulty #1: As soon as you download the file the remote server deletes it. You cannot affect this behaviour. The file you downloaded in the first step is the only copy you get.
Added difficulty #2: Commercially available software only. No writing a custom app that does all this in one step.

The problem: At no point may any of the files in this process be saved unencrypted to disk, ever. Saving them encrypted is heavily frowned upon but could potentially be acceptable.

I'm looking for solutions.

Obvious solution #1: RAM drive. Have the OS treat a hunk of RAM as really fast disk, save the files and do all the processing there.
Obvious solution #2: Encrypted folder. Each user (oh, did I mention multiple users?) has a passphrase to open it, nothing is stored outside it.
Obvious solution #3: Virtual machine. All storage and processing is done on a VM, which trashes itself without writing "to disk" when closed.

Any other ideas? Anything you'd prioritise trying?

Comments

[pappy-legba.livejournal.com]

Man, whoever writes the IT policy for ISIS is a dick.

[theweaselking.livejournal.com]

ISIS?

[lafinjack.livejournal.com]

The spy agency the Archer cartoon.

[lafinjack.livejournal.com]

Encrypted folder was something like what I was thinking, if it was allowed. Decrypt, unpack to encrypted folder, then you have a lot more flexibility from there. Is it being proofed on paper due to the encryption issue?

What happens if the original archive is damaged or incomplete? Another archive is generated at the other end?

[theweaselking.livejournal.com]

Writing to disk encrypted is disliked, it's just not "completely unacceptable".

Proofing: It's done on the computer first, but it's done AGAIN after printing because the nature of the printing process can expose errors that are not visible in the electronic copy, and the post-conversion documents are hellishly hard to read on a computer screen for most users. The exact details are not important - point is, the process is error-prone, and you can't lose the original until you're sure the output is good.

If the archive is bad: Someone gets called on the phone and a new one is generated on the far end. Same as if you delete it or something needs to be reprinted at a later time. Those are bad events that shouldn't happen, though.

[thornae.livejournal.com]

Does "no writing unencrypted files to disk, ever" include the swap partition?

I don't have anything resembling a better solution here, I'm just intrigued by how thoroughly Cold War paranoid it all seems.

[lafinjack.livejournal.com]

...the printing process can expose errors that are not visible in the electronic copy...

Oh man, bane of my existence when I worked at a print shop.

[lafinjack.livejournal.com]

That would be a RAM disk, no?

[thornae.livejournal.com]

Well, depends on your definition. I thought theweaselking was meaning to use a large amount of physical RAM as a disk - basically a very small DIY SSD, which isn't exactly elegant but gets around the "no writing to HDs" restriction.

I was referring to the chunk of actual hard drive that the OS uses as extra, slow RAM for stuff that it's not really concentrating on, like the page file in Windows systems. If that's included in the restriction, it's an extra level of wacky (but not entirely unjustified) paranoia, and correspondingly extra difficult to implement.

[lafinjack.livejournal.com]

Oh, gotcha. Yeah, I meant physical RAM, and your mention of the physical swap disk threw me because I assumed that would be covered under the "Thou Shalt Not Use Permanent Storage" commandment.

[thornae.livejournal.com]

Yah, I wasn't sure how restrictive that was. Although, given the other conditions, I should probably assume the worst.

[strawberryfrog.livejournal.com]

There is a difference between ram disk and an SSD - data on the SSD will still be there if you pull out the power cable.

[theweaselking.livejournal.com]

Unclear - but the machine has enough RAM that turning off the page file is totally doable.

[theweaselking.livejournal.com]

Yes. And the same is true of the windows page file, or the swap partition of a linux machine. That's "extra RAM" that's stored on the physical disk.

The comparison to the SSD is mostly in terms of relative size and speed - a RAM disk is crazy fast compared to spinny crap.

[jerril]

You also need a decryption and unpacking process that doesn't write files to the *system* temp folder in the middle of the process (which is different from the folder where you saved your encrypted archive). That is something that I don't think off-the-shelf software does... Just a nasty thought that occurred to me.

[jerril]

I'm just intrigued by how thoroughly Cold War paranoid it all seems.

That's not a terribad analogy - it's the right mindset for these standards anyways. They could be anyone, anywhere!

[theweaselking.livejournal.com]

PGP doesn't, but who knows what kind of crap closed-source SW is doing?

Alternately: Truecrypt the entire disk, and accept that someone physically on site will be forevermore required for all reboots.

[pappy-legba.livejournal.com]

Anyone who buys Apple's crap about Retina displays- "The human eye cannot detect any detail finer than 300-odd PPI" or whatever- has never been anywhere near a print shop.

[rbarclay.livejournal.com]

It's a nice little problem.

Your solution with a RAM disk obviously relies on the system never going down, for whatever reason, amd if the original file's that important, data loss should be, too. So I'd argue non-volatile storage as inevitable.
Then there's data integrity, so I'd argue multiple copies on different physical media next, and then on different machines in different locations.

And then I'd encrypt the whole systems, with the downside of no unattended reboots.

[theweaselking.livejournal.com]

Non-volatile storage, and replacement in case of loss due to power failure etc, are all THE OTHER END'S problem. The problem is that we, the printer, need to keep no recoverable copies of the data. They, the data owner, have it as much as they want and we can request a re-send by calling them up. However, "calling them up" cannot be part of the normal process, only the exception-handling process.

[theweaselking.livejournal.com]

I should be clear: As soon as we have a good printed-and-proofed copy, we delete the original and the transformed versions. We delete *all the copies* and mail out the physical.

[nsanity-au.livejournal.com]

When you say

"At no point may any of the files in this process be saved unencrypted to disk, ever. Saving them encrypted is heavily frowned upon but could potentially be acceptable."

What vector are you (or the client) attempting to protect against?

Physical lifting of your drives? (Hi, I just pulled the drives from the Server and i'm going somewhere else)

or just peeky pokey network snooping?

Finally just how is it possible that you're worried about digital copies and not physical ones?

In addition, how important is it that this service works? reliability wise? Does the client have the ability to resend a document?

I implemented a FTPS -> SFTP intermediary host at one point between a major airline and Mastercard US. We had some pretty crazy security requirements (ASIO T4 essentially) - they did random shit like encrypt the file, send over a VPN via FTPS and SFTP - with physical smart card based certs.

We had to secure erase our disks after the onward transfer.

[theweaselking.livejournal.com]

What vector are you (or the client) attempting to protect against?

Network snooping and "undelete" are the big ones. Physical theft is a secondary concern.

Finally just how is it possible that you're worried about digital copies and not physical ones?

Oh, we're worried about the physical copies, but
a) there's already procedures in place to handle that,
and
b) that's not my department.

Does the client have the ability to resend a document?

Yes. It's not completely trivial (we have to phone them and have a live person log in and click something) but it's not hard. RARELY losing the file is perfectly acceptable - it just has to be rare.

[skiriki.livejournal.com]

Hmm. I'd probably go with encrypted folder route, possibly on a separate internal HD that can be scrubbed and reformatted once the job is done, so it holds nothing but this sensitive, encrypted folder as long as the file is needed.

[andrewducker]

RAM Drive does seem like the most sensible step.

I take it no actual people will have direct access to the drive? Otherwise they can just copy files off if they feel like industrial espionage.

[theweaselking.livejournal.com]

There are people who require access to the data, in order to perform the process/print/proof parts. Their access is audited, they have security clearances, and there are procedures in place to prevent them from just walking off with it, but yes, the human is still a weak point. That's okay - the purpose here is to make the human the ONLY weak point.

[skington.livejournal.com]

If not for difficulty #2, and if the files were small enough, I'd suggest writing an app that could hold all of the intermediate steps in memory - although you might still have a problem when it came to printing, as print jobs are probably going to be stored on disc somewhere unless you're prepared to reinvent the wheel and send raw postscript over a serial cable.

Given difficulty #2, though, you're going to be using separate apps for (a) decrypting the archive, (b) unpacking it, (c) converting files, and (d) printing them. As such, each app is going to need to take input from the file system.

Encrypted folder on a RAM drive sounds like the easiest, if the files are small enough to fit entirely within half of the spare RAM (I say half because presumably the decryption and unpacking processes will work on copies of the files on the filesystem, and produce temporary scratch files.)

[theweaselking.livejournal.com]

print jobs are probably going to be stored on disc somewhere unless you're prepared to reinvent the wheel and send raw postscript over a serial cable.

Due to the nature of the printer and mitigating policies this is not an issue. For perfectly valid[1] technical reasons that wheel has long since been re-invented.

presumably the decryption and unpacking processes will work on copies of the files on the filesystem, and produce temporary scratch files

The decryption, no. It's built like that. The unpacking, possibly, but the unpacked files are encrypted and the temp files go into the archive's location, which will be in ram if we go with this plan..


[1]: No, really. Details are spoilers.