noise-0.0.0.0: Usable security for the internet, Deux

Copyright(c) Austin Seipp 2014
LicenseMIT
Maintaineraseipp@pobox.com
Stabilityexperimental
Portabilityportable
Safe HaskellSafe-Inferred
LanguageHaskell2010

Crypto.Noise.Tutorial

Contents

Description

Noise is a suite of cryptographic protocols similar in spirit to NaCl's crypto_box, or network solutions like TLS, but simpler, faster, with higher-security elliptic-curve cryptography, and stronger guarantees about deniability and identity hiding.

This protocol has many favorable security and usability properties, including:

Sender forward secrecy
After encryption of a Noise box, only the recipient can decrypt it (the sender cannot).
Deniable
The recipient of a Noise box can authenticate the sender, but cannot produce digitally-signed evidence binding the sender to anything.
Identity hiding
Noise boxes reveal no information about the sender or recipient to a 3rd-party observer.
High speed
Noise usues high-speed curves and ciphers designed by Dan Bernstein.
Padded
Noise ciphertext can be padded to avoid leaking plaintext lengths.
Built on "Encrypt-then-MAC" authenticated encryption
Any tampering with ciphertext will cause the recipient to reject the ciphertext prior to decryption.

Noise pipes are built on Noise boxes and designed for interactive communications, and in addition to the above, Noise pipes offer the following benefits:

Full forward secrecy or full key erasure
Compromise of any long-term private keys never compromises old pipes. But furthermore, compromise of an active endpoint in an ongoinng communication does not compromise prior ciphertexts either - a Noise pipe forgets its chain secrets upon every message.
Resistance to key-compromise impersonation
Even with a compromised private key, the compromised party can still verify other parties' identities in a Noise pipe.
Efficient, encrypted handshakes with short roundtrip
Handshakes allow clients and servers to communicate after only one round trip, offering room for validation checks or certificates.

This package offers:

A high-level box API
Boxes are created using the simple seal and open primitives. Boxes are encrypted, authenticated, and optionally anonymous. Furthermore, boxes are forward secret: only the receiver (identified by the receiving public key) can open them.
A simplistic networking API
Noise pipes can be utilized easily on top of TCP sockets using the connect and serve primitives. This makes it easy to write networking services with transparent encryption support, built on familiar send and recv primitives.
A high-level io-streams API
The networking API internally is built off a high-level API based on io-streams, which makes encrypting data over a pipe as easy as reading/writing to an InputStream or OutputStream. This also integrates with the networking API, making it easy to layer in extra transformations (compression, a high level packet format, etc).
X509 certificate support
This package makes it easy for noise pipes to automatically validate X509 certificates as part of the handshake. Parties can begin exchanging data after only one round trip. In this case, the certificate is offered, which is validated before continuing.

For more information visit https://github.com/trevp/noise/wiki.

Synopsis

Introduction

The noise package defines two sets of APIs: boxes and pipes. Boxes handle standalone messages, and pipes encrypt communication channels.

To begin, a sender and a receiver must create a keypair:

sender(senderPK, senderSK)       <- createKeypair
receiver(receiverPK, receiverSK) <- createKeypair

Send the public keys around, and keep the private keys safe.

Box API

Boxes are created using seal, and opened using open:

>>> b <- seal (Just sender) receiverPK 32 $ pack "Hello world!"
>>> print $ open receiverSK (Just senderPK) b
Just "Hello world!"

When creating a box, you specify the sending keypair, the receiving public key, the amount of random padding you want (to obscure the plaintext length), and the message. To open it, you specify the secret key of the receiving party, and the public key of the sender.

Attempting to open a box from someone other than the sender will result in failure.

Senders may also be anonymous, where the sender does not specify a long-term key pair:

>>> b <- seal Nothing receiverPK 32 $ pack "Hello world!"
>>> print $ open receiverSK Nothing b
Just "Hello world!"

In the above example, the sender of the box is anonymous without a keypair, and attempting to use a value other than Nothing as the key will error. When the sender is anonymous, they are only identified by a short-term ephemeral key, which is used only once for the corresponding box.

Once you have encrypted a value using seal, it can only be decrypted by the receiving party with the secret key. This property means that boxes are forward secret: once you are done creating them and have 'forgotten' the message, you cannot recover it. Furthermore, boxes are deniable: a recipient of a box can authenticate the sender. But they cannot produce signed evidence binding the sender to anything. Finally, boxes do not produce any evidence of who created them or who the receiver is, and resist tampering with a strong MAC.

network API

noise provides a simple, high-level networking API by default that closely mimmicks the traditional network API, but is more convenient to use as it will control closing sockets, and handle exception handling.

To start a server in the most simple case, all you need is the serve primitive, with a default configuration:

serve (Host "127.0.0.1") "9123" defaultPipeConfig $ \(ctx, remoteAddr) -> do
  putStrLn $ "Noise connection established from " ++ show remoteAddr
  send ctx $ pack "Hello world!"

Next, you can connect a client and use the familiar send and recv primitives to talk over the handle:

connect "www.example.org" "9123" defaultPipeConfig $ \(ctx, remoteAddr) -> do
  putStrLn $ "Connection established to " ++ show remoteAddr
  msg <- recv ctx
  putStrLn $ "Got message: " ++ show msg

You're done! Both ends are completely encrypted. Furthermore, the server can accept many incoming concurrent connections.

In the above example (and by default), pipes are anonymous like Noise boxes - parties are only identified by short-term ephemeral keys. This is the default configuration in the above example, with no extra options needed. Unlike boxes (which obviously require a key to open), both sides of a Noise pipe can function anonymously, or only one of them can (servers or clients). In all cases, pipes are authenticated to the specified parties keys only (long term or ephemeral). This makes Noise as easy to use as an unencrypted TCP socket (and easier than TLS even), while offering considerably better security in all aspects.

Note that without long-term keys, you can still be affected by a Man-in-the-Middle attack - attackers can intercept your packets, and give you a short-term ephemeral key claiming to be another party. But unlike protocols such as CurveCP, Noise does not require long-term identies for services, nor clients: identification and authentication is flexible (as we'll see later).

Anonymous clients and servers do have a bonus: they cut down on the required number of key exchanges. Underneath, Noise uses a "Triple Diffie-Hellman" agreement that does a 3-way DHE exchange (this 3DHE construction is the secret behind Noise's deniability, simplicity, and power). If either side of a particular pipe is anonymous, this count is reduced to two or even one during the initial handshake.

If you'd like to authenticate clients or servers with long-term public keys, you only need to specify this in the configuration, after sharing public keys however you want (over the phone, in meatspace, or with an Instagr.am picture). First, generate the long-term keys:

(public, private) <- createKeypair

Keep the private key somewhere safe. It's only 32 bytes long, so it isn't exactly huge. Distribute the public key to whoever needs it.

Once you've done that, just specify the keys in your configuration. For example, to specify the long-term keys for the server:

let sconf = defaultPipeConfig
              { confKeypair = Just (public, private)
              }

and use serve as usual.

Next, make the the client specifies the server's long-term public key in its connection:

serverPublicKey <- readFile "keys/ServerKey.pk"
let cconf = defaultPipeConfig
              { confExpectedKey = Just serverPublicKey
              }

Any attempt to connect to a server not identified by this public key will be rejected.

You can also do this in reverse: a client can have a long-term key and a server will only accept connections from this key. Likewise, any client connections not identified by this key are rejected:

let sconf = defaultPipeConfig
              { confExpectedKey = Just clientPublicKey
              }

let cconf = defaultPipeConfig
              { confKeypair = Just (public, private)
              }

Naturally, combining the two results in a fully authenticated tunnel.

Note that if a server expects a particular client identified by a long-term key, in most cases little point in using serve as it continuously serves connections. If you want single connections, you can use the listen and accept primitives instead.

io-streams API

The networking API is powered underneath by the io-streams API, which you can use to build your own Noise pipes over any InputStream and OutputStream. This transformation takes existing streams, does the negotiation over them, and returns the resulting encrypted streams.

A noise pipe comes in two flavors: a server pipe, and a client pipe. At the end of the day, there isn't really a difference between the two: it's just a matter of having the two duplex channels. To create a server pipe, first specify your PipeConfig, then use pipeServerStream, given an InputStream and OutputStream:

(decryptedInput, encryptedOut) <- pipeServerStream defaultPipeConfig (in, out)

Given the two streams in and out, pipeServerStream returns two streams which when read from/written to, encrypt/decrypt the specified data, and write it to the original streams. In other words, pipeServerStream transforms streams which take encrypted data and turns them into streams which take unencrypted data.

Note that every individual value which you write down the OutputStream, an individual message is created and written for it.

For example, with the above combinator, you can turn a Socket into an encrypted set of streams using socketToStreams:

createEncServerSocket :: Socket -> PipeConfig -> IO (InputStream, OutputStream)
createEncServerSocket socket conf
  -- 'socket' is the listening socket of a server.
  (in, out) <- (socketToStreams >=> pipeServerStream conf) socket
  return (in, out)

Now any ingoing/outgoing traffic is transparently encrypted.

Now we can do the same with a client pipe, in the other direction:

createEncClientSocket :: Socket -> PipeConfig -> IO (InputStream, OutputStream)
createEncClientSocket socket conf
  -- 'socket' is the socket connected to the server
  (in, out) <- (socketToStreams >=> pipeClientStream conf) socket
  return (in, out)

With these two combinators, we can turn any already connected sockets into encrypted tunnels. This is very close to how the Network API in this package is implemented, in fact.

Once you have the underlying streams, you can also naturally layer in other streams. For example, to layer in a compression stream:

zlibStreams :: Int -- ^ Compression level
            -> (InputStream ByteString, OutputStream ByteString)
            -> IO (InputStream ByteString, OutputStream ByteString)
zlibStreams l (input, output)
  = (,) <$> decompress input
        <*> compress (CompressionLevel l) output

-- Create an tunnel where any data is first compressed, then encrypted.
createZlibEncServerSocket :: Socket -> PipeConfig -> IO (InputStream, OutputStream)
createZlibEncServerSocket socket config = do
  -- 'socket' is the listening socket of a server.
  (in, out) <- (socketToStreams >=> pipeServerStream conf >=> zlibStreams 5) socket
  return (in, out)

In the above example, the socket is first turned into two streams, and then into an encrypted tunnel. Then, the zlibStreams function tranforms it so that any data is first compressed before encryption. After this, you can layer in any amount of other transformations you want: for example, your own networking protocol, or simply connecting the streams.

Note that when using pipes, the negotiation over the streams happens immediately: any waiting will cause the thread issuing the handshake to block, so if you want to handle this without waiting, be sure to use a timeout or the 'async' package.

Additionally, once you make a connection with connect or serve, you can pull out the InputStream and OutputStream to compose or transform it like above, using pipeContextStreams.

Padding

All messages in a noise pipe may optionally be padded to help prevent plaintext length analysis on the encrypted pipe. By default, boxes offer no padding - while pipes offer a small amount of random padding, but only in the initial negotiation phase!

To control the amount of padding set in a Noise box, use the input parameter to seal:

pad <- generateRandomNumberBetween 0 512
b <- seal Nothing receiver pad msg
...

This will encrypt the box and pad it with a random amount of data (between 0 and 512 bytes, assuming a good distribution for the number generator).

To pad the initial boxes that are exchanged in a Pipe handshake, you can set the confPadding parameter of the PipeConf. For example, setting confPadding = 32 will pad the initial handshake message you send with 32 random bytes.

Encrypting an active Noise pipe is a little more work. In essence, you must write a function to transform an OutputStream of this type:

f :: OutputStream PipeMsg -> IO (OutputStream ByteString)

There is an alias for this type - PipeMsgPadder.

The type PipeMsg is a simple message type, pairing a ByteString with an integer - the number of bytes to pad the message containing that buffer. You can think of OutputStream PipeMsg as being equivalent to OutputStream (ByteString, Int)

The simplest transformation is constantPadStream, which pads every message with a constant number of random bytes. This is used internally by default if you don't specify your own transformation:

constantPadStream :: Word32 -> PipeMsgPadder
constantPadStream n inp = makeOutputStream $ \v ->
  case v of
    Nothing -> write Nothing inp
    Just x  -> write (Just (PipeMsg x n)) inp

For example, constantPadStream 0 is a routine which will never pad any outgoing messages.constantPadStream 32 is a routine which will pad every message with 32 random bytes.

By default, a Noise pipe uses constantPadStream 0 internally when you do not specify your own padding routine.

In addition, there is also another routine provided - randomPadStream - which will pad every outgoing message with a randomly selected number of bytes (within the specified range).

Either end of a noise pipe can have their own unique padding routines specified without problem.

To specify the padding routine, simply specify it in the confPadder of your configuration. For example, to pad every message from the client with a random number of bytes in the range of 0 to 32, and pad every message from the server with 64 random bytes, you can write:

let sconf = defaultPipeConfig
              { confPadder = Just (constantPadStream 64)
              }

let cconf = defaultPipeConfig
              { confPadder = Just (randomPadStream 0 32)
              }

Note that padding is, of course, not free: as the bytes are taken from /dev/urandom, in some very unscientific benchmarks, a parameter like (randomPadStream 0 4096) cut the speed of encrypting/decrypting /dev/zero and writing it to /dev/null by 50% (200MB/s vs 100MB/s), with the Linux kernel's internal cryptographic routines dominating runtime profiles.

Initial message support

Noise pipes have an advantage over TLS in that only one round trip is needed to start sending data - as part of the handshake, the server and client can exchange data inside boxes.

By default, servers and clients are configured to send 16 bytes of zeros in their boxes, and they both validate the boxes have zeros in the handshake. This can be changed by updating PipeConfig before connecting.

This feature allows you to immediately exchange data (taking full advantage of the early roundtrip for your application), or do more exotic things like certificate validation (see below for X509 support).

Using this support is easy. In the PipeConfig for both parties, you can specify two fields:

 1 - confInitialMsg      :: Maybe (IO ByteString)
 2 - confInitialResponse :: Maybe (ByteString -> IO Bool)

These two fields specify the initial handshake procedure (in the order they occur). By default, the handshake looks like:

Step #1: Client ---- Ephemeral Key ---> Server (BEGIN)

Step #2: Client <--- noise_box #1  ---- Server (Initial server message)

Step #3: Client ---- noise_box #2  ---> Server (Initial client message)

Step #4: Client <---- ciphertext -----> Server (Bidirectional communication)

The initial server message in step #2 is specified with confInitialMsg on the server end, and the client checks if this is valid using confInitialResponse. If it is, the client responds with the result of its own confInitialMsg in step #3, and the server checks this message with its confInitialResponse. If this finishes, then the connection is established in step #4 (see Crypto.Noise.Protocol for details).

Note that even if these parameters are set to Nothing, validation always takes place: if you do not specify any override, or use Nothing explicitly, then the server uses the implicit default: a payload of 16 zero bytes each way, with the validation confirming this.

This means that if you override one of the initial messages, you MUST override the counterpart for the other party. This prevents clients and servers from finishing the handshake if they're incorrectly configured on either end. In other words, this is the default operation if you don't specify your own overrides:

confInitialMsg      = Just $ return (replicate 16 0x0)
confInitialResponse = Just $ x -> return (x == replicate 16 0x0)

X509 support

While public keys are powerful and small, many times they are not convenient. There is sometimes a need for an outside authority to verify integrity - in some cases, it's often far more convenient to have an authority sign certificates of authenticity for an endpoint, instead of distributing multiple public keys for each endpoint to all clients.

Like mentioned above, Noise pipe authentication is also flexible - as both ends of a pipe can be anonymous, it's possible to use different authentication mechanisms to establish secure connections.

To this end, this package features simple, built in X509 validation using the one-roundtrip initial box discussed previously. This can be used to verify a server presenting a certificate to a client is signed by a trusted CA in the users Certificate Store.

The default routines use the system certificate store. To enable the certificate, specify the offerX509 in the PipeConfig.

> let sconf = defaultPipeConfig { confInitialMsg = Just (offerX509 "noise.crt") }

where "noise.crt" is your certificate, signed by someone in the trusted store.

Next, make sure your client uses validateX509 in its PipeConfig:

> let cconf = defaultPipeConfig { confInitialResponse = Just $ validateX509 (Just "HOSTNAME") }

where "HOSTNAME" is the FQDN specified in the certificate.

That's it. Your client will automatically check the X509 certificate chain when they connect, and they'll reject invalid server certificates. This can also work the other way around: servers can validate client certificates too. Combined with anonymous ends, This allows you to easily use different authentication methods as opposed to public key distribution for Noise pipes.

You can also exercise more control over the validation check, using the more extensive validateX509_ function. For example, when testing, it's often useful to use a self-signed certificate. You can do this quite easily by using your own X509 ValidationCache. First, create a fingerprint of your self-signed certificate:

$ openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -out noise.crt
...
$ openssl x509 -noout -sha256 -in noise.crt -fingerprint
SHA256 Fingerprint=<YOUR_FINGERPRINT>

(where <YOUR_FINGERPRINT> is the SHA256 fingerprint of your certificate).

Next, set up your ClientConfig to use the exceptionValidationCache, specifying the service identifier and the fingerprint:

> let fp1 = Fingerprint $ B8.pack "YOUR_FINGERPRINT"
      validCache = exceptionValidationCache
        [ (("YOUR-FQDN", B.empty), fp1)
        ]

      cconf = defaultPipeConfig { confInitialResponse = Just $ validateX509_ (Just "HOSTNAME") (Just validCache) Nothing }

Done! Now your clients will have an exception only for this fingerprint and no other self-signed certificate will work. Alternatively, using tofuValidationCache you can perform "Trust On First Use" (TOFU) validation as well, making Noise negotiations even more flexible.

Alternatively, you can override the failure check for the validation procedure. For example, if you want to accept any self-signed certificate (WHICH IS A BAD IDEA, MIND YOU):

> let selfSigned x = return (x == [] || x == [SelfSigned])
      cconf = defaultPipeConfig { confInitialResponse = Just $ validateX509_ (Just "HOSTNAME") Nothing (Just selfSigned) }

This new configuration overrides the failure check: if there are no errors OR the certificate fails only due to self-signing, then the certificate is accepted.

Differences from the standard

Currently, this package mostly implements the Noise specification faithfully.

The large exception is that we use Curve25519 instead of Curve41417. Curve41417 offers a 200-bit security level, which is really what Noise aims to offer (NaCl crypto_box routines similarly only offer 128-bit security). As a result, keys are only 32 bytes as opposed to 52 bytes.

In the future, this protocol will use Curve41417 by default.

Other notes

Here are some important things to note when using this protocol:

You can't turn off the encryption
There is no way to disable encryption. Furthermore, the primitives should be high-speed enough to not negatively impact large workloads.
No protection against traffic analysis by default
While the initial boxes feature randomized padding by default, the outgoing ciphertext is not padded in any way.. If you want to randomize your traffic a bit, use the randomPadStream transformer in your configuration to automatically pad outgoing traffic. The underlying random number generator is MWC seeded from /dev/urandom, so it should be fast, and pick numbers in the specified range with a good distribution.