SugarDaddy

Hello. Here's a quick breakdown of my problem. Any help would be greatly appreciated.

I have an application written in MFC that communicates via HTTP with a server running PHP 5. No problems there. However, the communication needs to be encrypted for reasons that are unimportant. On the server, a PHP script generates a unique identifier (using libmcrypt) that is used as a session key for the encryption / decryption. The session key is then encrypted using Triple DES, base64 encoded, the initialization vector attached to the front, then base64 encoded again. The client's username and password is used as the key to encrypt the session key.

Great, so that all works. On the client side is where the problem occurs. I am using Microsoft's CryptoAPI to do the decryption. After base64 decoding and parsing out the initialization vector and generating the decryption key, the decryption process fails. CryptoAPI's CryptDecrypt function shoots back NTE_BAD_DATA. The CryptDecrypt function has a flag used in block encryption/decryption that says whether the plain text to decrypt is the final block or not. Being that I'm not using block encryption, I set that flag to TRUE. The documentation for NTE_BAD_DATA says that when that flag is TRUE, then the padding is wrong. So I set the flag to FALSE and it decrypts without fail. However, the result is not correct. What's going on? I'm not crypto expert, so I'm hoping someone around here is.


Pseudo of what I'm doing to decrypt:

So it's basically by the book. One thing to note is that the doc for CryptDecrypt says "Decrypts data previously encrypted with CryptEncrypt"

That seems pretty weak to me if CryptDecrypt can ONLY decrypt data encrypted with CryptEncrypt. Shouldn't it be based on the algorithm, not the implementation? I would think so, but then I realize that it's Microsoft and I start to doubt that reasoning.

Any help would be greatly, greatly appreciated.

mackenga

Well, I'm not a crypto expert exactly but since there are no replies...

Frankly, I'm boggling about the notion that the only block doesn't decrypt correctly when treated as the 'last block'. I've boggled for a few minutes now, and bear in mind this is more general principles I'm thinking about rather than inside knowledge of the specific process in question, but here's my best shot.

If it uses padding, the block encryption/decryption expects a particular block size. This full size would be occupied by your ciphertext on a non-final block, but partially occupied by padding on a final block. So when you say it's the final block, the padding is wrong, but when you don't say so, the padding couldn't be since there shouldn't be any. What might be wrong, though, is the size of the block. I mean, if it normally expects blocks a certain size, with the final block padded as necessary, then it may not be sure how to react to your arbitrary sized single block.

Given it as a non-final block, I don't know what it would do. It might just assume that this is the block size, however many bytes it's been given; is there any way to /set/ the block size? This could be a clue; do you need to explicitly tell it not to work with multiple blocks? If the function that did the encrypting wasn't expecting to be sending standard-sized blocks, maybe it didn't pad it; if the function you're using to decrypt assumes it always gets padding and always deals with blocks of a fixed size, maybe you need to tweak the other end to get it to play the game; why not produce a fixed-size block at the other end, if this is possible, just to go easy on the MS code? (How often have we all had to do something like this? "X is working, it's Y's fault, but we need to change X, because it's Microsoft Y and will probably only be fixed by stealth in five years' time.")

If the system is expecting, say, 1K blocks to pluck a number out of the air, and you give it a 512 byte complete ciphertext and tell it it's a non-final block, then judging by the behaviour you describe what it's probably doing is reading past the end of the buffer to get its 1K just because it expects them to be there. This would give it the wrong ciphertext, hence the wrong plaintext but wouldn't set any alarm bells ringing if you're just passing in a pointer, and wouldn't cause any serious misbehaviour from the program since it'll only be reading through the pointer anyway, and not way off in some random location.

Like I say, I'm no expert so that's all I can really suggest. Hope this is better than nothing.