Ludovic Rousseau's blog

Monday, May 25, 2020

10 years of blogging

I started this blog 10 years ago, in April 7th 2010.
Since then I wrote about many aspects of the smart card use in GNU/Linux and macOS.

Statistics

Some statistics about the number of articles per year.
Years 2010 and 2020 represent only half a year of activity.

Conclusion

I do plan to be present and continue in the next 10 years.

GitHub Sponsors: first payment

Since January 2020 I am part of the Github sponsors program. See my previous article: GitHub Sponsors.

Payment

I just got my first payment in May 2020 for the amount of €66.25. Yeah!

The next payment should occur in June 2022, in 2 years. Unless new sponsors arrive in the meantime.

Use of the money

That is not a huge amount of money but that will help pay for the VPS I rent at OVH to host my projects at https://muscle.apdu.fr/. They are mostly pcsc-lite and libccid.

Conclusion

I have 2 active options to send me money:

this Github sponsor program
bitcoin

Feel free to use whatever to prefer.

Friday, May 15, 2020

New PyKCS11 1.5.8 available

I just released a new version of PyKCS11, a Python wrapper above the PKCS#11 API.
See "PyKCS11 introduction" or "PyKCS11’s documentation".

The project is registered at Pypi: https://pypi.org/project/PyKCS11/

Changes:

1.5.8 - May 2020, Ludovic Rousseau

CKA_ALWAYS_AUTHENTICATE is boolean
CKM_VENDOR_DEFINED_...

Fix name: use CKM_ instead of CKR_ prefix
Use an explicit hex prefix: CKM_VENDOR_DEFINED_0x45

Add missing CKM_*, CKA_*, CKF_*, CKD_*, CKK_*, CKN_*, CKO_*, CKR_* from PKCS#11 v3.0
fix test_asymetric.py for RSA_PSS_Mechanism

Friday, May 8, 2020

Your PC/SC application 200% faster

Dana Keeler reported an issue on pcsc-lite at "SYS_USleep in SCardEndTransaction in winscard_clnt.c causing slowness in Firefox".

The problem was initially reported on Firefox "firefox is very slow and crashes because of p11-kit-proxy.so" and also on p11-kit "firefox is very slow and crashes because of p11-kit-proxy.so".

The issue

The pcsc-lite performance issue comes from this piece of code in SCardEndTransaction:

 /*
  * This helps prevent starvation
  */
 randnum = SYS_RandomInt(1000, 10000);
 (void)SYS_USleep(randnum);

For each call to SCardEndTransaction() you will get a delay between 1 ms and 11 ms. So on average you get 6 ms of delay each time.

Performance measures

I wrote a Python program to get the duration of 100 calls to SCardEndTransaction():

#! /usr/bin/env python3

from smartcard.scard import *
from smartcard.pcsc.PCSCExceptions import *
from time import time

hresult, hcontext = SCardEstablishContext(SCARD_SCOPE_USER)
if hresult != SCARD_S_SUCCESS:
    raise EstablishContextException(hresult)

hresult, readers = SCardListReaders(hcontext, [])
if hresult != SCARD_S_SUCCESS:
    raise ListReadersException(hresult)
print('PC/SC Readers:', readers)
reader = readers[0]
print("Using reader:", reader)

hresult, hcard, dwActiveProtocol = SCardConnect(hcontext, reader, SCARD_SHARE_SHARED, SCARD_PROTOCOL_ANY)
if hresult != SCARD_S_SUCCESS:
    raise BaseSCardException(hresult)

nb = 100
total = 0
for i in range(nb):
    hresult = SCardBeginTransaction(hcard)
    if hresult != SCARD_S_SUCCESS:
        raise BaseSCardException(hresult)

    before = time()
    hresult = SCardEndTransaction(hcard, SCARD_LEAVE_CARD)
    if hresult != SCARD_S_SUCCESS:
        raise BaseSCardException(hresult)
    after = time()
    delta = after - before
    print(delta)
    total += delta
print("total: {} ms".format(total * 1000))
print("average: {} ms".format(total * 1000 / nb))

hresult = SCardDisconnect(hcard, SCARD_LEAVE_CARD)
if hresult != SCARD_S_SUCCESS:
    raise BaseSCardException(hresult)

hresult = SCardReleaseContext(hcontext)
if hresult != SCARD_S_SUCCESS:
    raise ReleaseContextException(hresult)

Results

As expected the random delay is of 6 ms (or 0.006 second) on average.

The history

I don't know why this delay is needed. I went into the previous versions of the source code file PCSC/src/winscard_clnt.c but the oldest version (from March 2002, 18 years ago) already has this delay.

The code was written David Corcoran, the initial author of pcsc-lite. It was my very early days in the pcsc-lite project at that time.

The solution

I don't see any good reasons to have a delay here. I guess it was to solve a problem with the communication between pcscd and libpcsclite at that time. The communication mechanism has been redesigned in version 1.6.0 (May 2010) and the delay should now be useless and even problematic as we saw.

My solution is then to remove the problematic code. The was done in this commit.

The results

I used again my Python program to measure the performances of SCardEndTransaction(). I now have:

The mean delay is 9.5x10^-6 s so 9 µs or 0.009 ms or 0.000009 second. The speedup factor is huge: x647.
The function is now 600 times faster than before.

You can note a high decrease on the 4 first values. My guess is that it is an effect of CPU memory caches in action. I have not investigated this point. This is left as an exercise for my readers.

Macro benchmark

It is nice to have a huge improvement in one PC/SC function but does that help real applications?

For a real smart card application I used OpenSC with a standard command: list all the objects of a smart card.
I used a very simple shell script:

#!/bin/bash

for i in {1..100}
do
 pkcs11-tool --list-objects
done

Slow smart card

First I used a very old smart card I have in my collection: a Gemplus GPK 8000 card.
Since the card is very old and slow the shell script will do 10 rounds instead of 100.

what	time (s)
Before	23.34
After	22.84

Speedup: x1.02 or 2%

The gain is very limited. This is because the card is so slow that the benefit from the new SCardEndTransaction() is negligible.

Fast smart card

I then used a much faster smart card: a Yubikey 5 from Yubico. It is not a real smart card but a token with a CCID interface and a chip that understand APDU commands.
Since the device is fast I used 100 rounds of pkcs11-tool.

what	time (s)
Before	9.85
After	3.02

Speedup: x3.26 or 226%

This time the gain is highly visible. The pkcs11-tool command is now 3 times faster.

Results

You will get a high acceleration with fast smart card.
I was able to get a full execution of pkcs11-tool 3 (three) times faster than before the change.

I was really impressed by this result. For almost 10 years pcsc-lite was slow and could be improved by just removing 2 lines of code.

Potential regression?

I don't think this change will create a regression and will break existing code.

I made a beta version of pcsc-lite including the change available at http://ludovic.rousseau.free.fr/softwares/pcsc-lite/pcsc-lite-1.8.26-05d48e5.tar.bz2.
Please test this version with your PC/SC application. In case of a problem with PC/SC transactions then open an issue at Salsa https://salsa.debian.org/rousseau/PCSC/-/issues or github https://github.com/LudovicRousseau/PCSC/issues.

I plan to wait a few weeks to get potential feedback before I make a new release of pcsc-lite.

Conclusion

pcsc-lite is now much faster.

This happened because someone complained that Firefox was slow and someone at Mozilla investigated the issue to find a problem in pcsc-lite.

I am very happy to use Free Software programs where it is possible and easy to find problems in software you use.

Friday, May 1, 2020

PCSC sample in Free Pascal (Lazarus)

Here is a new PCSC sample in Free Pascal language I promised in PC/SC sample in different languages.

Lazarus

From Lazarus web site:

What is Lazarus?
Lazarus is a Delphi compatible cross-platform IDE for Rapid Application Development. It has variety of components ready for use and a graphical form designer to easily create complex graphical user interfaces.

Infintuary Pascal PC/SC Sample

I found a Pascal PC/SC Sample code for PCSC with Lazarus at http://infintuary.org/stpcsc.php.

This program is also available at https://github.com/ccy/pcsc in a more recent version. Unfortunately the more recent version has Windows specific code and can't be used on GNU/Linux. I reported the issue at Fix build on GNU/Linux: CheckOSError() is for Windows.

License

The license is custom but could be enough. From http://infintuary.org/stpcsc.php

The Pascal PC/SC Sample is freeware and can be used for any purpose.
By downloading the Pascal PC/SC Sample you agree to the terms of use.

The terms of use is different from the license available on the github project. For example the terms of use does not explicitly allow modification of the source code, and the use is allowed only for legal purpose. In general it is a bad idea to re-invent a new license text.

Installation

You copy the files MD_Events.pas, MD_PCSC.pas, MD_PCSCDef.pas, MD_PCSCRaw.pas and MD_Tools.pas.

Source code

unit main;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls,
  MD_PCSCRaw, MD_PCSCDef, MD_Tools;

type

  { TForm1 }

  TForm1 = class(TForm)
    Button1: TButton;
    Memo1: TMemo;
    procedure Button1Click(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure AddLogMemo(Msg: string);
  private
    FPCSCRaw: TPCSCRaw;

  public

  end;

var
  Form1: TForm1;

implementation

{$R *.lfm}

{ TForm1 }

procedure TForm1.Button1Click(Sender: TObject);
var
  PCSCResult: Dword;
  hContext: THandle;
  SizeReaders: LongWord;
  pReaders: PChar;
  hCard: LongInt;
  dwActiveProtocol: Cardinal;
  pioSendPCI, pioRecvPCI: pSCardIORequest;
  inBuffer : TBytes;
  outBuffer: TBytes;
  outSize: Cardinal;
  i: Integer;
  outString: String;
begin
    FPCSCRaw := TPCSCRaw.Create;
    FPCSCRaw.Initialize;

    // Establish context
    PCSCResult := FPCSCRaw.SCardEstablishContext(SCARD_SCOPE_SYSTEM, nil, nil, hContext);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardEstablishContext succeeded.')
      else AddLogMemo('SCardEstablishContext failed: ' + PCSCErrorToString(PCSCResult));

    // List readers
    PCSCResult := FPCSCRaw.SCardListReaders(hContext, nil, nil, SizeReaders);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardListReaders succeeded.')
      else AddLogMemo('SCardListReaders failed: ' + PCSCErrorToString(PCSCResult));

    GetMem(pReaders, SizeReaders);
    PCSCResult := FPCSCRaw.SCardListReaders(hContext, nil, pReaders, SizeReaders);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardListReaders succeeded.')
      else AddLogMemo('SCardListReaders failed: ' + PCSCErrorToString(PCSCResult));

    // Use the first reader
    AddLogMemo('Using: ' + pReaders);

    // Connect to the card
    hCard := -1;
    dwActiveProtocol := 0;
    PCSCResult := FPCSCRaw.SCardConnect(hContext, pReaders, SCARD_SHARE_SHARED,
      SCARD_PROTOCOL_Tx, hCard, dwActiveProtocol);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardConnect succeeded.')
      else AddLogMemo('SCardConnect failed: ' + PCSCErrorToString(PCSCResult));

    // Send Select Applet command
    pioRecvPCI := nil;
    if dwActiveProtocol = SCARD_PROTOCOL_T0
      then pioSendPCI := @SCARDPCIT0
      else pioSendPCI := @SCARDPCIT1;

    inBuffer := HexStringToBuffer('00 A4 04 00 0A A0 00 00 00 62 03 01 0C 06 01');
    outSize := 258;
    SetLength(outBuffer, outSize);
    PCSCResult := FPCSCRaw.SCardTransmit(hCard, pioSendPCI, Pointer(inBuffer),
      length(inBuffer), pioRecvPCI, Pointer(outBuffer), outSize);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardTransmit succeeded.')
      else AddLogMemo('SCardTransmit failed: ' + PCSCErrorToString(PCSCResult));

    SetLength(outBuffer, outSize);
    AddLogMemo('Received (' + IntToStr(outSize) + ' bytes): ' + BufferToHexString(outBuffer));

    // Send test command
    inBuffer := HexStringToBuffer('00 00 00 00');
    outSize := 258;
    SetLength(outBuffer, outSize);
    PCSCResult := FPCSCRaw.SCardTransmit(hCard, pioSendPCI, Pointer(inBuffer),
      length(inBuffer), pioRecvPCI, Pointer(outBuffer), outSize);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardTransmit succeeded.')
      else AddLogMemo('SCardTransmit failed: ' + PCSCErrorToString(PCSCResult));

    SetLength(outBuffer, outSize);
    AddLogMemo('Received (' + IntToStr(outSize) + ' bytes): ' + BufferToHexString(outBuffer));

    outString := '';
    for i := 0 to outSize -3 do
      outString := outString + chr(outBuffer[i]);
    AddLogMemo(outString);

    // Disconnect
    PCSCResult := FPCSCRaw.SCardDisconnect(hCard, SCARD_LEAVE_CARD);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardDisconnect succeeded.')
      else AddLogMemo('SCardDisconnect failed: ' + PCSCErrorToString(PCSCResult));

    // Release context
    PCSCResult := FPCSCRaw.SCardReleaseContext(hContext);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardReleaseContext succeeded.')
      else AddLogMemo('SCardReleaseContext failed: ' + PCSCErrorToString(PCSCResult));
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
  Memo1.Clear;
end;

procedure TForm1.AddLogMemo(Msg: string);
begin
  Memo1.Lines.Add(Msg);
end;

end.

Remarks

You will have to create a Form with a TMemo widget and a TButton widget. This part of the solution is not described here.

You click on the button and the program executes.

Output

Comments

High level API

I used the low level API provided in file MD_PCSCRaw.pas. These are direct equivalent of the WinSCard API. But a higher level API is also available and is provided in the file MD_PCSC.pas.

Unfortunately I was not able to find a way to get the list of connected readers with the high level API. Maybe I missed something (I am very new to Free Pascal). I reported the issue at make TPCSC.GetPCSCReaderList() a public method.

64-bits and GNU/Linux issue

A much more important issue is that this wrapper does not work on a 64-bits GNU/Linux system. I had to install a Debian system on a i386 CPU to write and test my code.

The problem is that the wrapper uses a Free Pascal type THandle to store the SCARDCONTEXT and SCARDHANDLE types. The problem is that a THandle in 32-bits on a 64-bits CPU but SCARDCONTEXT and SCARDHANDLE are 64-bits types on a 64-bits CPU using GNU/Linux.

The problem comes from the definition of long (DWORD) on a 64-bits CPU. On GNU/Linux a long is 64-bits. But on Windows a long is 32-bits only. If you want to have a 64-bits variable on Windows you need to use the long long type. This is because Linux is LP64 and Windows is LLP64. See 64-bit data models for more details.

The wrapper should work on macOS even on a 64-bits CPU. This is because Apple does not use any DWORD in the WinSCard API but int32_t instead.

I reported the problem at Using THandle for SCARDCONTEXT and SCARDHANDLE is wrong on GNU/Linux.

Windows only?

As I already wrote in the introduction the, more recent, code available at https://github.com/ccy/pcsc is for Windows only. Some work is needed to make it available (again) for macOS and GNU/Linux.

Conclusion

If you use the Free Pascal language and you use Windows then this wrapper can help you.
If your system is not Windows or you want portability on diffrent systems then this wrapper may not be a good choice.

Wednesday, April 22, 2020

SCARD_ATTR_CHANNEL_ID and USB devices

Sometimes you need a way to identify which USB device is the smart reader reported by PC/SC.
That was the request of Stephan Guilloux in issue 68 and Pull Request 69.

SCARD_ATTR_CHANNEL_ID

The WinSCard API already provides something like that with the SCARD_ATTR_CHANNEL_ID attribute.

The Microsoft documentation says:

SCARD_ATTR_CHANNEL_ID

DWORD encoded as 0xDDDDCCCC, where DDDD = data channel type and CCCC = channel number:

The following encodings are defined for DDDD:

0x01 serial I/O; CCCC is a port number.

0x02 parallel I/O; CCCC is a port number.

0x04 PS/2 keyboard port; CCCC is zero.

0x08 SCSI; CCCC is SCSI ID number.

0x10 IDE; CCCC is device number.

0x20 USB; CCCC is device number.

0xFy vendor-defined interface with y in the range zero through 15; CCCC is vendor defined.

So for a USB device the DDDD value is 0x0020 and we have 2 bytes for the CCCC value.
We decided to use the 2 bytes to encode the bus number in the most significant byte (MSB) and the device address in the least significant byte (LSB).

Implementation

I added the support of this new attribute in the CCID driver version 1.4.32.

I also added support in the PCSC Unitary Test SCardGetAttrib.py to check the code is working correctly.

Example (Low Level Python API)

#! /usr/bin/env python

from smartcard.scard import *
from smartcard.pcsc.PCSCExceptions import *
from struct import unpack

hresult, hcontext = SCardEstablishContext(SCARD_SCOPE_USER)
if hresult != SCARD_S_SUCCESS:
    raise EstablishContextException(hresult)

hresult, readers = SCardListReaders(hcontext, [])
if hresult != SCARD_S_SUCCESS:
    raise ListReadersException(hresult)

for reader in readers:
    hresult, hcard, dwActiveProtocol = SCardConnect(hcontext, reader,
        SCARD_SHARE_DIRECT, SCARD_PROTOCOL_ANY)
    if hresult != SCARD_S_SUCCESS:
        raise BaseSCardException(hresult)

    print("reader:", reader)
    hresult, attrib = SCardGetAttrib(hcard, SCARD_ATTR_CHANNEL_ID)
    if hresult != SCARD_S_SUCCESS:
        print(SCardGetErrorMessage(hresult))
    else:
        print(attrib)
        # get the DWORD value
        DDDDCCCC = unpack("i", bytearray(attrib))[0]
        DDDD = DDDDCCCC >> 16
        if DDDD == 0x0020:
            bus = (DDDDCCCC & 0xFF00) >> 8
            addr = DDDDCCCC & 0xFF
            print(" USB: bus: {}, addr: {}".format(bus, addr))

    hresult = SCardDisconnect(hcard, SCARD_LEAVE_CARD)
    if hresult != SCARD_S_SUCCESS:
        raise BaseSCardException(hresult)

hresult = SCardReleaseContext(hcontext)
if hresult != SCARD_S_SUCCESS:
    raise ReleaseContextException(hresult)

The important part of the code is the line:

        DDDDCCCC = unpack("i", bytearray(attrib))[0]

As documented by Microsoft, the SCARD_ATTR_CHANNEL_ID attribute does not return a buffer of bytes but a DWORD.
The difference is that the order of the 4 bytes of the DWORD is important and depends on the CPU architecture. The code must take into account the endianess of the CPU. Since the Python program does not know if the CPU is little or big endian we must use unpack() to let the CPU decode the 4 bytes as an integer using its internal endianess.

If you have a better solution please share it.

Output

$ python3 SCardGetAttrib.py 
reader: Gemalto PC Twin Reader 00 00
[8, 1, 32, 0]
 USB: bus: 1, addr: 8
reader: Cherry KC 1000 SC Z [KC 1000 SC Z] 01 00
[5, 1, 32, 0]
 USB: bus: 1, addr: 5

Example (high level Python API)

We can also use the higher level Python API.
The code is much shorter but may be more complex to understand and translate into a PC/SC wrapper in another language.

#! /usr/bin/env python

from smartcard.System import readers
from smartcard.scard import (SCARD_SHARE_DIRECT, SCARD_ATTR_CHANNEL_ID)
from struct import unpack

for reader in readers():
    print("reader:", reader)
    card_connection = reader.createConnection()
    card_connection.connect(mode=SCARD_SHARE_DIRECT)

    attrib = card_connection.getAttrib(SCARD_ATTR_CHANNEL_ID)
    print(attrib)
    DDDDCCCC = unpack("i", bytearray(attrib))[0]
    DDDD = DDDDCCCC >> 16
    if DDDD == 0x0020:
        bus = (DDDDCCCC & 0xFF00) >> 8
        addr = DDDDCCCC & 0xFF
        print(" USB: bus: {}, addr: {}".format(bus, addr))

Output

Of course the output is the same.

$ python3 getAttrib.py 
reader: Gemalto PC Twin Reader 00 00
[8, 1, 32, 0]
 USB: bus: 1, addr: 8
reader: Cherry KC 1000 SC Z [KC 1000 SC Z] 01 00
[5, 1, 32, 0]
 USB: bus: 1, addr: 5

Use case

The use case of this new feature is be able to make a bijective relation between a PC/SC smart card reader and a USB device connected to the host.

The information returned by SCARD_ATTR_CHANNEL_ID is also returned by the lsusb command (on GNU/Linux).

$ lsusb 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 006: ID 1bcf:0005 Sunplus Innovation Technology Inc. Optical Mouse
Bus 001 Device 005: ID 046a:00a4 Cherry GmbH 
Bus 001 Device 008: ID 08e6:3437 Gemalto (was Gemplus) GemPC Twin SmartCard Reader
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

You can see that the PC/SC reader "Gemalto PC Twin Reader 00 00" is at "USB: bus: 1, addr: 8" for PC/SC and at "Bus 001 Device 008:" for lsusb.

You can also ask lsusb to list only this specific device using:

$ lsusb -s 1:8
Bus 001 Device 008: ID 08e6:3437 Gemalto (was Gemplus) GemPC Twin SmartCard Reader

You may have noticed that the device "Bus 001 Device 005" is the PC/SC reader "Cherry KC 1000 SC Z [KC 1000 SC Z] 01 00" and it is also my keyboard. It is a Cherry KC 1000 SC Z.

More details

You can also get more information about the USB reader at the PC/SC level using CM_IOCTL_GET_FEATURE_REQUEST to get PCSCv2_PART10_PROPERTY_wIdVendor and PCSCv2_PART10_PROPERTY_wIdProduct values.
See "Identifying a reader model (part 2)".