Using GPG to Encrypt Files and Data

GPG is mostly used for securing email. However it can be used to encrypt arbitrary files and data using public-key encryption algorithms. (It can also encrypt files using symmetric ciphers and passwords, but that is not discussed here.)

GPG 1.X series is designed more for servers. GPG 2.X series is designed more for desktops and provides S/MIME support. They are just different and 2.X is not better than 1.X. This example uses gpg 1.4.

Make the Directory

For this example we don't want to mess up your real keys, so we'll make an alternate directory and use it for everything.

mkdir ./test
chmod 0700 ./test

Generate Keys

Since gpg is hyper-paranoid this might be 10 minutes.

gpg --homedir ./test --gen-key

You'll then answer a lot of questions:

• Algorithm:  RSA and RSA
• Keysize: for RSA,  2048 bits is fine.  The NIST says this is good until 2030.
• Expiration: Does not expire.  I haven't played around with expired keys to see how they work.
• Real Name:  use something descriptive or your name depending on how you are going to use this "Encryptor Robot" (ok bad example)
• Email Address:  put something that looks sorta an email  "robot@encryptor.town"
• Comment:  go nuts!

Finally, it's your choice on if you need a password or not. If this is for bulk encryption, you probably don't want a password.

Then wait.........  If you are using a VM, perhaps login with a different window and/or copy some files in or out of it.

At the end of this you should be able to list and export the keys

$ gpg --home ./test --list-keys
./test/pubring.gpg
------------------
pub   2048R/02A9B20A 2012-04-07
uid                  Encryptor Robot <robot@encryptor.town>
sub   2048R/14C5AD94 2012-04-07

Figuring out your parameters

GPG has a complicated system of determining the compressions algorithm and level, the preferred symmetric cipher and the preferred hash function. It's designed for email systems where the sender and receiver might have different capabilities and preferences. It could come from the operating system, a personal configuration file, preferences in the key, or from the command line. For automated systems this can get in the way, and in fact make it difficult to determine what is being used when. Typing in:

gpg --verbose --version

should show something like the following:

Supported algorithms:
Pubkey: RSA, RSA-E, RSA-S, ELG-E, DSA
Cipher: 3DES (S2), CAST5 (S3), BLOWFISH (S4),
        AES (S7), AES192 (S8), 
        AES256 (S9), TWOFISH (S10),
        CAMELLIA128 (S11), CAMELLIA192 (S12), 
        CAMELLIA256 (S13)
Hash: MD5 (H1), SHA1 (H2), RIPEMD160 (H3), SHA256 (H8), SHA384 (H9), 
      SHA512 (H10), SHA224 (H11)
Compression: Uncompressed (Z0), ZIP (Z1), ZLIB (Z2), BZIP2 (Z3)

You can decide what's best for in terms of compression, but as of April 2012, the recommended setup is to use AES and SHA256. There may be specific requirements for your industry that might over-rule this for everything else this is just fine.

Now we have two ways to set the preferred algorithms. The easiest and clearest is doing this explicitly on the command line. That is shown in the examples below using --cipher-algo AES --digest-algo SHA256. The other way is by modifying the key's preferences. Type in gpg --homedir ./test --list-secret-keys. You get something like this:

./test/secring.gpg
------------------
sec   2048R/02A9B20A 2012-04-07
uid                  Encryptor Robot 
ssb   2048R/14C5AD94 2012-04-07

Using that ID in bold, type in gpg --homedir ./test --edit-key 02A9B20A. Now you'll get an interactive prompt.

# see verbose defaults
showpref

# set short defaults
pref

# set to AES/SHA256/NoCompress
# change NoCompress to whatever you like (Z1,Z2,Z3)
setpref S7 H8 Z0

# see your work
# notice how 3DES and SHA1 and just hardwired in
showpref

# byebye
quit

Encrypt and Decrypt

The examples below use stdin and stdout. You can specify an output file using --output. For input files, just add the filename as the last command line argument.

# encrypt
# --homedir ./test == what key database to use
# -q --no-tty --batch --yes == make gpg be a silent as possible
# -encrypt --armor == encrypt and make nice ascii format
# --trust-model always means to disable the "web of trust" stuff which
#    may or may not make sense in an automated environment
# --recipient == what key to use.
echo "client9.com" | gpg -q --no-tty --batch --yes \
 --homedir ./test \
 --encrypt --armor \
 --trust-model always  \
 --cipher-algo AES \
 --digest-algo SHA256 \
 --compress-algo uncompressed \
 --recipient robot@encryptor.town > /tmp/file.gpg

# decrypt
# This is simpler.  It figures everything out from the message 
# and key database
gpg --homedir ./test -q --no-tty --batch --yes --decrypt < /tmp/file.gpg

Export and Import

To export and import. I'm not sure what the standard suffix is for keys. I've seen ".gpg" and ".key" used. (Comments welcome.)

# export public
gpg --home ./test --armor --export robot@encryptor.town \
    > /tmp/robot-public.gpg
# export private
gpg --home ./test --armor --export-secret-key --armor \
    > /tmp/robot-private.gpg

# in new directory with gpg databases:
gpg --import public-or-private-key.gpg

openssl public key encryption

If one wants to encrypt aribtrary data or a file using public key encryption algorithms, one can use OpenSSL and S/MIME.

For algorithms and key sizes, the NIST Computer Security Division says 2048-key for RSA and 128-bit key for AES is just fine until 2030 (Special Publication 800-57, page 66).

The examples below all use stdin and stdout.  You can use files by using "-in INFILE -out OUTFILE" on the command line.


# create 2048 bit key
openssl req -x509 -nodes -newkey rsa:2048 \
    -keyout PRIVATE1.pem \
    -out PUBLIC1.pem \
    -subj '/'


# encrypt, note that it is different each time
echo "client9.com" | \
   openssl smime -encrypt -aes128 -binary -outform DEM PUBLIC1.pem | \
   base64
echo "client9.com" | \
   openssl smime -encrypt -aes128 -binary -outform DEM PUBLIC1.pem | \
   base64


# and decrypt
echo "client9.com" | \
  openssl smime -encrypt -aes128 -binary -outform DEM PUBLIC.pem | \
  openssl smime -decrypt -binary -inform DEM -inkey PRIVATE.pem 


# let's create another key
openssl req -x509 -nodes  -newkey rsa:2048 \
    -keyout PRIVATE2.pem \
    -out PUBLIC2.pem \
    -subj '/'


# 
# we can now encrypt with *both public keys*, 
# and then use *either private key* to decrypt
# (can use as many keys as you want)
#
echo "client9.com" | \
   openssl smime -encrypt -aes128 -binary -outform DEM \
   PUBLIC1.pem PUBLIC2.pem | \
   base64


# let's use key #1 to decrypt
echo "client9.com" | \
   openssl smime -encrypt -aes128 -binary -outform DEM \
   PublicCert1.pem PublicCert2.pem | \
   openssl smime -decrypt -binary -inform DEM  -inkey PRIVATE1.pem


# let's use key #2 to decrypt
echo "client9.com" | \
   openssl smime -encrypt -aes128 -binary -outform DEM \
   PublicCert1.pem PublicCert2.pem | \
   openssl smime -decrypt -binary -inform DEM  -inkey PRIVATE2.pem 


# snazzy!

Using Raw RSA

I wrote this article since the first few Google results for "openssl public key encryption" all used raw RSA.  Eeek.  If your data is always smaller than the key size (minus some overhead), you could use raw RSA but must make sure you use OAEP padding schemes (the older PKCS padding is probably ok, but maybe not given you have a "small input" probably with a known structure).  Do not try and create some RSA-CBC hybrid beast.  Use S/MIME instead.

# generate keypair

openssl genrsa -out private.pem 2048

# extract public key

openssl rsa -in private.pem -out public.pem -outform PEM -pubout

# decrypt

echo "client9.com" | \

   openssl rsautl -encrypt -oaep -inkey public.pem -pubin | \

   openssl rsautl -decrypt -inkey private.pem

# please check the maximum input size first!

RSA Key Size Recommendation: 2048 bits

The NIST Computer Security Division in Special Publication 800-57 on page 66 recommends using 2048-bit RSA keys until 2030.

stringencoders now has memory-free query-string tokenizer

stringencoders has a new function modp_qsiter .  It's a URL query string tokenizer in C.  Why another one?  Many libraries have dependencies on a data structures library, a specialized string library or worse do memory allocation for you, typically causing overhead and needless copying.  Not this one.  It's up to you do make the copy into whatever string library you want, or put it into a STL map<stirng,string> or not (and define behavior when a key is duplicated).

One a related note, modp_burl.h also has a new function modp_burl_decode_raw which is just like modp_burl_decode except that "+" is preserved (compare to PHP url decode and rawurldecode) 

More detail are from modp_qsiter.h:

/**

* Query string key value pair iterator.  Uses no heap, makes no copy, makes

*  no modification of input.  Think of this as a super-strtok_r.  This

*  also does not do query-string un-escaping.

*

* qsiiter_t qsi;

* const char* qs = "foo=bar&ding=bar";

* qsiter_reset(&qsi, qs, strlen(qs));

* while  (qsiter_next(&qsi)) {

*    // we only get start and length of key,value

*    // up to you how to copy it or not, on heap or stack

*    // with strcpy, strncpy, strndup, memcpy, mempcpy, strlcpy, whatever

*    // callers job to alloc/free memory

*

*    const char* key = (const char*) malloc(qsi.keylen + 1);

*    strcpy(key, qsi->key, qsi->keylen);

*    const char* val = (const char*) malloc(qsi.vallen + 1);

*    strcpy(val, qsi->val, qsi->vallen);

*    printf("key = %s, value = %s\n", key, val);

*    free(key);

*    free(value);

* }

*

*

*/

gcc unused attribute

Dusting off my C skills and wanted to silence some benign used parameter warnings

warning: unused parameter ‘len’ [-Wunused-parameter]

Props to Martin Pool's blog for this nice macro:

#ifdef UNUSED 
#elif defined(__GNUC__) 
# define UNUSED(x) UNUSED_ ## x __attribute__((unused)) 
#elif defined(__LCLINT__) 
# define UNUSED(x) /*@unused@*/ x 
#else 
# define UNUSED(x) x 
#endif

Use it as such:

int foobar(int a, int UNUSED(b))

It's nothing fancy or unique or even not commonly known, but I had to look it up. And so I'm giving Martin some SEO love.

Google Safe Browsing without The Browser

My work at Etsy involving Google Safe Browsing has now been open-sourced on github. I also wrote about it on Code As Craft. Enjoy!

Accessibility versus Security

Steve Yegge via Rip Rowan's post:

But I'll argue that Accessibility is actually more important than Security because dialing Accessibility to zero means you have no product at all, whereas dialing Security to zero can still get you a reasonably successful product such as the Playstation Network.