I'm trying to validate a DKIM email signature using PKC Tools 4.0 with some success using the RSADemo project as a starting point.

All is well following the steps [Load] [Sign] and [Verify] in sequence.

So far, I've only managed to make the signature verification work properly when the TstRSAKey object is freshly generated or loaded with the private key; however, in the case of using DKIM on and email server, the receiver only has access to the public key string (retrieved from DNS record), content and the content signature (retrieved from an email header).

How should I set up the TstRSAKey object with the public key, etc, so that I can call VerifySignature(...)? My attempts thus far using RSAKey.LoadPublicKeyFromStream() have not been successful.

Thanks,
Todd

If you only have the RSA public key, use the TstRSAPublicKey component.

The public key in the DNS record is supposed to correspond to the base64 encoding of a PKCS#1 encoded RSA public key. To create a file that corresponds to a pre-existing key format, either Base64 decode the blob to a file you load using PKCS1-RSAPublic, or add the "-----BEGIN RSA PUBLIC KEY-----" and "-----END RSA PUBLIC KEY-----" PEM header and footer to the base64 encoding, and use PKCS1-PEM-RSAPublic.

I should have also mentioned that I have also tried loading the file both ways (using the code snippet in the prior post), i.e. both as base64-decoded/raw rsa public key, and as base64-encoded with the suggested header/footer, without success.

This is the PEM-formatted public key I used which was generated at https://www.socketlabs.com/domainkey-dkim-generation-wizard/

-----BEGIN RSA PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDdhL+h3wgj4a3Cdby4B3a4AZVg
XVEeamkgjINW9UtCQOz717biJ7SqOSmvk8cmPgbzNqnbAb/TOQQzIGzvsGtHp5Ts
MR9/xmDJyV6sMkkb0tu96IwS7FFDJlIcKqqdMOqDL1HTcFfCw5+XGEWynXHFRB2z
kUolV1jrwmiCjN27WQIDAQAB
-----END RSA PUBLIC KEY-----

...Just trying to get more information out in front.
Thanks again
Todd

Looks like my second post did not make it somehow.  Here's the code snippet:

  with stRSAPublicKey1 do
  begin
    Name := 'stRSAPublicKey1';
    EnabledDesignTime := False;
    EnabledRunTime := False;
    EncryptEncoding := eeEME_PKCS1_v1_5;
    SignEncoding := seEMSA3;
    MGFHashAlgorithm := haSHA1;
    // PublicKeyFormat: PUBLICKEYBLOB-RSA-SIGN,PUBLICKEYBLOB-RSA,PUBLICKEYBLOB-RSA-KEYX,RSA-XmlString-Public
  end;

  MS := TMemoryStream.Create;
  try
    MS.LoadFromFile('c:\temp\pubsave.txt');
    MS.Position := 0;
    if not stRSAPublicKey1.LoadPublicKeyFromStream('', MS) //PKCS1-RSAPublic? PKCS1-PEM-RSAPublic?
      then Raise Exception.Create('LoadPublicKeyFromStream failed');
  finally
    MS.Free;
  end;

As I wrote above, if the public key file has that format, the PublicKeyFormat should be PKCS1-PEM-RSAPublic. The statement you might be looking for is

PublicKeyFormat.SelectedOptionNames := 'PKCS1-PEM-RSAPublic';

Great, that makes sense, except that I can't find any tDSIIFPublicKeyFormat-derived implementation for PKCS1-RSAPublic or PKCS1-PEM-RSAPublic in my downloaded source.

I'm thinking, similar to RSA-XmlString-Public and others, I should see something like this somewhere in the source?

class function tRSA<...>Format.RegisterIdentifier: string;
begin
  Result := 'PKCS1-PEM-RSAPublic';
end;

Right, there is supposed to be a stPkcs1PublicFormat unit in PKC 4.0, but it appears to be missing. Fixing.../|\-

Great! Thanks!

Should I look for the fix in Downloads?

I don't see anything new there yet, but perhaps too soon to expect?
Todd

It will be available in Downloads as a hotfix, once it has been updated to the StreamSec.DSI.* arc. Expect it Monday June 19.

Okay, thanks again

Uploaded.

Thanks yet again! I have this working now with only one additional comment/question.

As far as I can tell, LoadPublicKeyFromStream() requires that the public key redundantly reside in a file defined by prop PublicKeyFileName owing to the check in LoadPublicKeyFromStream():
begin
...
    if Result then
      Result := lKey.E.Compare(MustGetPublicKey.E) and
                lKey.N.Compare(MustGetPublicKey.N);
...
end

*** which indirectly calls ...

function TstCustomMPKey.HasPublicKeyStream: Boolean;
begin
  Result := (fPublicKeyFileName <> '') and FileExists(fPublicKeyFileName);
end;

*** and subsequently reads the same public key that was passed in to LoadPublicKeyFromStream() from file using GetPublicKeyStream().

*** I added the code below to the RSA demo's "Create Key" button event to test the complete signing and verification of signature.  
 
procedure TfrmMain.btnCreateKeyClick(Sender: TObject);
...
    RSAKey.SignEncoding := seEMSA2;
    edtSign.Text := RSAKey.GenerateSignature(memMsg.Text,sfBase64);

>>>>>>>>>>>>>>>>>> not sure why this should be required >>>>>>>>>>>>>
    lSS.Position := 0;
    lSS.SaveToFile('c:\temp\p.txt');
    lSS.Position := 0;
    stRSAPublicKey1.PublicKeyFileName := 'c:\temp\p.txt';
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

    stRSAPublicKey1.PublicKeyFormat.SelectedOptionNames := 'PKCS1-RSAPublic-a';
    lSS.Position := 0;
    if not stRSAPublicKey1.LoadPublicKeyFromStream('', lSS) then // sets stRSAPublicKey1.SignEncoding = seEMSA2!
      begin
        lblSignStat.Caption := 'LoadPublicKey failed';
        exit;
      end;

    stRSAPublicKey1.SignEncoding := RSAKey.SignEncoding;

    if stRSAPublicKey1.VerifySignature(memMsg.Text,edtSign.Text,sfBase64) then
      lblSignStat.Caption := 'OK (verified)'
    else
      lblSignStat.Caption := 'Failure (NOT verified)';
...


I'm hoping that I'm just doing something wrong in my code which is causing this behavior, but otherwise seems like a "load-from-stream" type of call wouldn't also rely on identical content from a file.

I am not sure what you are asking, but you seEMSA2 is an obscure and obsolete sign encoding. Normally, you will want to use seEMSA3 (= PKCS#1 v1.5) or seEMSA4 (= PKCS#1 v2 = RSA-PSS-SSA). You should also set the sign encoding, digest algorithm and MGF algorithm *after* loading the public key. Some key formats might persist these properties and consequently change them on Load.

Will do on SignEncoding=seEMSA3/4.  The use of ESMA2 here was more of an artifact of testing, etc.

That said, the assignment of stRSAPublicKey1.SignEncoding does follow loading, and I'm setting it equal to SignEncoding for RSAKey.GenerateSignature(), but I neglected to explicitly set digest algorithm and MGF algorithm as you pointed out, so thanks for that.

But now I'll try to clarify the original question...

In the example above, if I remove the code:
>>>>>>>>>>>>>>>>>> not sure why this should be required >>>>>>>>>>>>>
    lSS.Position := 0;
    lSS.SaveToFile('c:\temp\p.txt');
    lSS.Position := 0;
    stRSAPublicKey1.PublicKeyFileName := 'c:\temp\p.txt';
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
then LoadPublicKeyFromStream('', lSS) returns false/fails.

It fails because, even though LoadPublicKeyFromStream() takes the argument lSS, it subsequently loads the exact same content again from file [PublicKeyFileName] in GetPublicKeyStream(). Given that LoadPublicKeyFromStream() is loading from a stream, it doesn't seem like it should be reading it from file as well.

The function LoadPublicKeyFromStream is actually one of the few inadequately named methods in ST 4.0. It is used by SSH components for verifying that the public key sent by the remote peer matches the one saved locally.

The name makes more sense for the TstKeyManager.LoadPublicKeyFromStream implementation, which persists the key in aStrm if it is left untouched by the active key filters (including PKI filters such as certificate chain filters and revocation filters).

Got it, thanks for the explanation. I can easily work around that if inclined.

So I have things working end to end now *within* PKC Tools, but now having trouble reconciling with other-software-created keys.

There are several websites that generate private/public key pairs for use with DKIM, and my next test was to load those using PKC Tools and then sign, validate, etc. This wasn't working so I tried going the other way, i.e. generate key pairs in PKC Tools and verify with openssl.

Using openssl I can load PEM files created by the online DKIM tools and extract the public key with the following command
c:\OpenSSL>openssl rsa -in example.com.priv -pubout

However, I have not been able to do this successfully with PKC Tools yet. In this case, I'm using your same demo program [Create Key] (1024), and using all available formats, though I'm thinking RIVATEKEYBLOB-RSA is probably the one that should work. Then base64 encode and break at 64 chars and enclose with -----BEGIN RSA PRIVATE KEY----- / -----END RSA PRIVATE KEY-----.

Here's a typical example of a failed attempt:

-----BEGIN RSA PRIVATE KEY-----
BwIAAAAEAABSU0EyAAQAAAEAAQD9avxBQhFOCE1wjP8LV4CNT3rNVkocq7XAKPAo
VCNIb8xgToX9k6fXJkOuGupHfO1WhLkpcrheDcvFK1zjsOwCXVe9M55TOgfaZUZc
BU3PnT1JE17c3CLFcCCYyvU3lODJvG8uzzyUIbd/nY8lLgbFh2gYoKWzmF776E39
jaKPll1BJzdyo7+xgFtuBrt/OVRwyJhrVzJA7k2zk0cCteW9YwS6J/PqYSqU+sfq
07FP1HBDf4SmNJXxbWdzsoEcSOkhFpvPETnpJcQm36HCzFO84p+Ku5Ke5yUM1D0+
SY8DbXEaZEkp5Bh2fkcbVghRTFhh+66BR5LC1OxtcFx+PDmlVb7Sv2hxvH0r8DvO
+CkU2W1CvPyXBKfncoOpAQ0NFW6fQWkSlpcj9/iDR2+jtsM4urEvf9kKVQKCw/HL
/PB3GUEg0MeuTJye9JXjiFFgTXUcNDZXZq164aUCsSSE1qPwaAzTrqTk6GFpBMWL
rZaoR5p4GY4G76sGkkQNQ34nMiUoRDuA8vgulU2Nr79EEgMQaKBkHYqHBQIlJ3O6
4VZO9JguG3RofI7U9tQgMTuk8glDOZIlvjFhzS0NnCjBR1gwIcwGtVbcbXHab6rw
bXcEGZsN1PV9k7XCCO2jFT3ObqARvhLAbDkgg1BPtwpU2UiqVawTItoPGf03vKRY
KXS4YQtRrzMJcLMdyx9/YFdwalI003dCCWaVoUOeil/ilELWIbb1B5cu4tHa4+0U
c9+f7udFCatab6beFtMB3LJ7AQA=
-----END RSA PRIVATE KEY-----


Also, at first glance, the code for tRSAPRIVATEKEYBLOBFormat.SaveToStream() doesn't seem to match what I find on the web regarding the prescribed record format.  To my untrained eye, in the code that I have, I can't find one that does.

I also note that the command "openssl genrsa 1024" and all online DKIM key gen tools always create a base64 sequence which begins with "MI" so I'm pretty sure I must be doing something wrong.

No. The OpenSSL "RSA PRIVATE KEY" format is the base64 encoding of the PKCS#1 private key format. This format is not supported by PKC Tools 4.0, but typically you do not need it. Private keys are supposed to stay private and should normally only be used by the software that generated them.

Oh right. That makes sense ;)

Thanks again for the guidance.

Another snag...
The using the public key produced using "PKCS1-RSAPublic-a", the DKIM validation site returned the following error:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Validating Signature

result = invalid
Details: public key: OpenSSL error: wrong tag
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

From your earlier post:
>>>>>>>>>>>>>>>>>>>>>>>>
... add the "-----BEGIN RSA PUBLIC KEY-----" and "-----END RSA PUBLIC KEY-----" PEM header and footer to the base64 encoding, and use PKCS1-PEM-RSAPublic.
<<<<<<<<<<<<<<<<<<<<<<<<
which is presumably what is produced by the missing file that you said was missing and then later provided named StreamSec.DSI.Pkcs1PublicFormat.pas (registers "PKCS1-RSAPublic-a") ?

But looks like DKIM expects a key in the form that correlates with "-----BEGIN PUBLIC KEY-----" and "-----END PUBLIC KEY-----".

The PKCS1-PEM-RSAPublic format is only included in the full ST 4.0 distribution.

The standard textual representation of DKIM records include the base64 encoding of a PKCS#1 RSA Public key.

Right, and according to our email discussion, I should be able to easily overcome this difference by simply encoding the DER output of the recently provided 'PKCS1-RSAPublic-a' formatter.

I did try that immediately upon receiving the new code, but that did not solve the problem. So yesterday I dug deeper into the Encode() logic called by tRSAPublicKeyFormat.SaveToStream(). Comparing the code in Encode() to the internal structure associated with BEGIN PUBLIC KEY (base64) as explained here //stackoverflow.com/questions/18039401/how-can-i-transform-between-the-two-styles-of-public-key-format-one-begin-rsa, it's clear that 'PKCS1-RSAPublic-a' does not produce the structure required by DKIM.

Specifically, PKCS1-RSAPublic-a does not include (using example from stackoverflow answer):
30 82 01 22          ;SEQUENCE (0x122 bytes = 290 bytes)
|  30 0D             ;SEQUENCE (0x0d bytes = 13 bytes)
|  |  06 09          ;OBJECT IDENTIFIER (0x09 = 9 bytes)
|  |  2A 86 48 86   
|  |  F7 0D 01 01 01 ;hex encoding of 1.2.840.113549.1.1
|  |  05 00          ;NULL (0 bytes)
|  03 82 01 0F 00    ;BIT STRING  (0x10f = 271 bytes)

Partly because I need to make progress, and partly to fully prove to myself that my understanding is correct, I created a new formatter which extends the logic provided in PKCS1-RSAPublic-a to include the missing elements. That is working now but I still need to implement in reverse for LoadFromStream().

That work will go a little faster now, but my understanding is that this was already supported by PKC Tools 4.0.
Todd

I see, and you are quite correct. There is indeed a subtle difference between "BEGIN PUBLIC KEY" and "BEGIN RSA PUBLIC KEY". The former signifies a X.509 subject public key structure. The latter signifies a PKCS#1 RSA Public Key structure, which corresponds to the output of PKCS1-RSAPublic-a.

I ran into another issue which seems worth mentioning...

I kept getting errors from the online DKIM checker sites "data too small/large for key size" which led me here:
//stackoverflow.com/questions/6658728/rsa-signature-size/6662179#6662179

So I streamed the result of RSAKey.GenerateSignature(Text,sfBase64) to file and noted that the base64 decoded file size was always 129/257 instead of 128/256. On closer inspection, I see that use of sfBase64 creates a string which when decoded using MIME64ToStr() includes a leading NULL byte.

I corrected the issue in my code using: b := StrToMIME64Str(RSAKey.GenerateSignature(S,sfBinary));

This might indeed very well be the case, and is something anyone has to watch out for every time. Every standard has its own very specific encoding rules. Sometimes a RSA signature is interpreted as a field element (in the residual field modulo N represented by non-negative integers in the range 0..N-1), sometimes as an integer (with 2's complement encoding of negative values). In the latter case a leading 1 bit signifies a negative number.

I think I'm close to the finish line, but I ran into one more issue.

I've been following along some published code in perl which signs and validates DKIM emails.

The step that's failing for me is the point at which I need to validate the canonicalized headers digest, using the public key and signature.

Using identical arguments, my code fails, I think for lack of a step before calling the VerifySignature().

In the case of the published functioning code (perl), the author performs an encryption step on the signature binary prior to calling $result = $rsa_pub->encrypt($signature);

Then he follows with compare $result to calculate_EM(...$digest...), etc, to validate.

Looks like I should be able to do the same except that Encrypt checks size and raises error 'Message too large'.

In the perl program the author also calls $rsa_pub->use_no_padding; just prior to encrypt which, according to my understanding, makes the result less safe (raw encryption?), but effectively relaxes this constraint.

Searching my StreamSec code, I see lots of references to block padding modes (e.g. bpmNone), but I don't see how to implement that for stRSAPublicKey1.Encrypt(b,'',sfBinary)

I suppose you have stumbled upon a misnomer in the perl library code. A RSA signature verification involves a "RSA Public Key Operation", which is also involved in a RSA Public Key Encryption operation. What is called "encrypt" in the perl library code, is most likely just a RSA public key operation.

Rather, in order to figure out why the signature verification fails, you should step debug the VerifySignature method call, down to the methods in unit StreamSec.DSI.IFC. (No, there is no option to turn on high resolution error reporting for this method. This is by design, because having such error reporting turned on at run time would open up any number of exploits. And at debug time you might just as well step debug.) At some point the code execution fails, and this will reveal the likely cause of the error.

For instance, if there is a failure at the initial bit length check a few lines into tDSIIF.SSASignatureVerification, it is either due to a string pre-processing failure of the signature (such as including too many characters in the string input to the base64 decoder, or some other base64 decoding failure), trying to decode a signature generated with a significantly larger RSA key, etc.

If there is a failure at the EMSA3Verification, step into that method. If the input here is pure garbage, begin by checking that the input to tDSIIF.SSASignatureVerification wasn't significantly shorter than the expected bit length (which might also indicate a string pre-processing failure) Otherwise, the signature was likely generated either using a different RSA key pair, or using RSA-PSS (EMSA4). If there is a digest size mismatch, you picked the wrong DigestAlgorithm. If the final digest comparison fails, you picked the wrong "to be signed" input, formatted it incorrectly, used the wrong string encoding, etc.

I definitely understand where you're coming from, but I've checked the in's extremely carefully and I'm pretty sure that the pubkey, digest and signature that I'm providing for VerifySignature are identical to values just before calling $rsa_pub->encrypt($signature).

So I went straight to openssl to see if I could replicate the perl behavior.  Looks like the following command gives me the equivalent "translation":

openssl rsautl -encrypt -raw -inkey pubkey-dkv.txt -pubin -in signature-dkv.der -out signature-dkv-enc.der

And yet, I also see that the perl program's general method of verification is different. (although possibly equivalent at some level?)
The following code is my Delphi conversion of the perl program. Ironically, you will probably understand what it's doing and why better than I do.

    b := MIME64ToStr(OSToBytes(Trim(edtVerSig.Text)));  // This is a string of the base64 encoded signature found in the DKIM b= tag.
    emLen := length(b);
    StrToFile(b,'c:\openssl\signature-temp.bin');  // write the binary signature to a file so I can call openssl to do the work
    WinExecAndWait32('c:\openssl\bin\openssl rsautl -encrypt -raw -inkey c:\openssl\pubkey-dkv.txt -pubin -in c:\openssl\signature-temp.bin -out c:\openssl\signature-temp-enc.bin', SW_HIDE);
    b := FileToStr('c:\openssl\signature-temp-enc.bin');   // read it back into b

    case stRSAPublicKey1.DigestAlgorithm of  // stRSAPublicKey1 not playing much of a role using this method, but using value of DigestAlgorithm for convenience
      haSHA1: T := HexToStr('3021300906052B0E03021A05000414');
      haSHA256: T := HexToStr('3031300d060960864801650304020105000420');
    end;
    T := T + FileToStr(edtVerContentFile.Text);  // This is a file containing the content that produced the signature
    tLen := length(T);
    if (emLen < tLen + 11) then
      begin
        raise Exception.Create('Intended encoded message length too short.');
      end;

    PS := StringOfChar(chr($ff),emLen - tLen - 3);  // prefix
    EM := chr(0) + chr(1) + PS + chr(0) + T;

    if EM = b then
      lblSignStat.Caption := 'OK (verified)'
    else
      lblSignStat.Caption := 'Failure (NOT verified)';

In any case, this code seems to get the job done, except that I don't want to call out to the OS to run openssl. So I guess if there's a better way to arrive at the same result using TstRSAPublicKey.VerifySignature, I'm happy to do that.

At this point, I'd be equally happy to have a way to replace the openssl command above with an equivalent from StreamSec.

Thanks yet again for your time - Todd

What you got there is EMSA3 verification (PKCS#1 v1.5). Are you sure the stRSAPublicKey1.SignEncoding equals seEMSA3?

Yes, pretty sure:

    with stRSAPublicKey1 do
    begin
      EnabledDesignTime := False;
      EnabledRunTime := False;
      PublicKeyFileName := 'c:\temp\p.txt';
      SignEncoding := seEMSA3;
      EncryptEncoding := eeEME_PKCS1_v1_5;
      DigestAlgorithm := haSHA1;
      MGFHashAlgorithm := haSHA1;
    end;


It returns false in tDSIIF.EMSA3Verification at...
    if Result then
      Result := CompareBytes(aDigest,aOffset,lT,J + 2 + lHashIDLen,aCount);

OK, in your own code you are not hashing the input, but treating it as a digest. If you already have a digest of the input, use TstRSAPublicKey.VerifySignedDigest, which expects the first input to be the digest of the ToBeSigned data, instead of the ToBeSigned data itself.

That did it! Many thanks!
Todd

This answer led me to symmetric method GenerateSignatureDigest() which solved my problem on the signing side as well.

Thanks again!

One final question (I hope)

Seems like I must be missing something obvious but... How do I programmatically set values for TstRSAKey.PublicKeyFormat.SelOptions?

Tried
    PublicKeyFormat.SetSelectedOptionNames(['RSA-XmlString-Public']);
and
    PublicKeyFormat.SelOptions := 'RSA-XmlString-Public,PUBLICKEYBLOB-RSA,PUBLICKEYBLOB-RSA-SIGN,PUBLICKEYBLOB-RSA-KEYX';

without success for different reasons.

You can do it either by calling the SetSelectedOptionNames method, or by assigning a string value to the SelectedOptionNames property (hence you got the first right, the second wrong).

Also note that your project must use any and all required units. At design time you can do this by invoking the IDE expert that suggests required units. If you start with pure run time code, you have to add those units manually.

The names of the required units are specific to the components and the formats you use. For instance, the components and the formats you specify above, might be satisfied by:

stAes,
stDrbgAesCtr,
StreamSec.DSI.CAPIFormat,
StreamSec.DSI.IFC,
StreamSec.DSI.NETFormat,
stSHA1,
stSHA256;

The reason these units aren't automatically used by the units implementing the formats and components, is because the framework has been implemented so that these units might optionally be replaced by alternative implementations, just by changing uses clauses.

Interesting, the RSA-XmlString-Public I was trying to use is registered in StreamSec.DSI.NETFormat, which I already had in my uses clause.

But looks like StreamSec.DSI.CAPIFormat are StreamSec.DSI.IFC are also required for RSA-XmlString-Public... I guess due to some upstream dependencies.

In any case, all is well now.

Thanks again -Todd