Important!

Blog moved to https://blog.apdu.fr/

I moved my blog from https://ludovicrousseau.blogspot.com/ to https://blog.apdu.fr/ . Why? I wanted to move away from Blogger (owne...

Thursday, May 28, 2020

Unicode characters in a reader name

It is now possible to use Unicode characters in a reader name.

History

Since the beginning of pcsc-lite (at least since the first version of pcsc-lite in 2002 that is in a Version Control System) only a subset of ASCII was considered as legal characters for a PC/SC reader name.

In 2011 I added the character ";" in the list so that it is possible to use the "&" sign (encoded as "&" since the reader list is encoded as XML in the Info.plist file). This was to support a reader name like "Giesecke & Devrient".

In 2012 I added the characters "[" and "]".

In 2020 I add support of any Unicode character.

This request came from the use of the reader name "SoloKeys Solo ๐Ÿ". See the Salsa ticket "Unicode in USB Product string not supported." for more details.

Demo

First example

± pcsc_scan 
Using reader plug'n play mechanism
Scanning present readers...
0: ู…ุฑุญุจุง ุจุงู„ุนุงู„ู… ๐Ÿ˜€ ๐ŸŽ‚ 00 00
1: ืฉืœื•ื ืขื•ืœื ๐Ÿ˜Ž ๐Ÿ˜ผ 01 00

Sat May 16 10:52:58 2020
Reader 0: ู…ุฑุญุจุง ุจุงู„ุนุงู„ู… ๐Ÿ˜€ ๐ŸŽ‚ 00 00
Event number: 1
Card state: Card removed,
Reader 1: ืฉืœื•ื ืขื•ืœื ๐Ÿ˜Ž ๐Ÿ˜ผ 01 00
Event number: 0
Card state: Card inserted,
ATR: 3B BE 96 00 00 41 03 00 00 00 00 00 00 00 00 00 02 90 00
In case you do not have the correct font installed in your web browser here is a picture version of the same output.

You can note that the reader names are reversed between the text version and the image version. I let you find what is the "problem" here.

Second example

± pcsc_scan 
Using reader plug'n play mechanism
Scanning present readers...
0: ๐Ÿ˜บ ๐Ÿ˜ธ ๐Ÿ˜น ๐Ÿ˜ป ๐Ÿ˜ผ ๐Ÿ˜ฝ ๐Ÿ™€ ๐Ÿ˜ฟ ๐Ÿ˜พ 00 00
1: ๐Ÿ’‹๐Ÿ’˜๐Ÿ’๐Ÿ’–๐Ÿ’—๐Ÿ’“๐Ÿ’ž๐Ÿ’•๐Ÿ’Ÿ๐Ÿ’”๐Ÿงก๐Ÿ’›๐Ÿ’š๐Ÿ’™๐Ÿ’œ๐Ÿ–ค 01 00
 
Sat May 16 11:17:00 2020
 Reader 0: ๐Ÿ˜บ ๐Ÿ˜ธ ๐Ÿ˜น ๐Ÿ˜ป ๐Ÿ˜ผ ๐Ÿ˜ฝ ๐Ÿ™€ ๐Ÿ˜ฟ ๐Ÿ˜พ 00 00
  Event number: 0
  Card state: Card removed, 
 Reader 1: ๐Ÿ’‹๐Ÿ’˜๐Ÿ’๐Ÿ’–๐Ÿ’—๐Ÿ’“๐Ÿ’ž๐Ÿ’•๐Ÿ’Ÿ๐Ÿ’”๐Ÿงก๐Ÿ’›๐Ÿ’š๐Ÿ’™๐Ÿ’œ๐Ÿ–ค 01 00
  Event number: 0
  Card state: Card inserted, 
  ATR: 3B BE 96 00 00 41 03 00 00 00 00 00 00 00 00 00 02 90 00
Again with the screen capture:

Of course you have no obligation to use some many funny Unicode characters in your reader name. It was just an example.

Availability

You need to use CCID version 1.4.33 or more and pcsc-lite version 1.9.0 or more.

These versions are not yet available (when I write this article) so I prepared snapshot of both software at http://ludovic.rousseau.free.fr/softwares/pcsc-lite/. They are pcsc-lite-1.8.26-047789c.tar.bz2 and ccid-1.4.32-e782d48.tar.bz2.

You can also use the current git version of pcsc-lite and ccid if you know what you do.

Linux

I made the development and tests on a Debian GNU/Linux system.

macOS

I also tested the new CCID driver on macOS Mojave and it works fine with Unicode characters. I found no issue.

Conclusion

I do not expect to see many smart card readers with emoticons, but maybe names with characters from non-Latin alphabets.

Monday, May 25, 2020

10 years of blogging

I started this blog 10 years ago, in April 7th 2010.
Since then I wrote about many aspects of the smart card use in GNU/Linux and macOS.

Statistics

Some statistics about the number of articles per year.
Years 2010 and 2020 represent only half a year of activity.


Conclusion

I do plan to be present and continue in the next 10 years.

GitHub Sponsors: first payment

Since January 2020 I am part of the Github sponsors program. See my previous article: GitHub Sponsors.

Payment

I just got my first payment in May 2020 for the amount of €66.25. Yeah!

The next payment should occur in June 2022, in 2 years. Unless new sponsors arrive in the meantime.

Sponsors

For now I have 3 sponsors:Martin Paljak, Jaroslav Imrich and CrazyMarvin. A big thank you to you!
The sponsor list is public. You can see it at https://github.com/sponsors/LudovicRousseau/.
 
They sponsor me for a total of $9/month. This number is NOT public but I want to be transparent with you. What you can see on my sponsor page is that I am "90% towards $10 per month goal". So after some mathematical calculation it is easy to get the $9/month.

Github also has the GitHub Sponsors Matching Fund. So half of that money comes from github/Microsoft. That is is first time I receive something from Microsoft ๐Ÿ˜€.

Use of the money

That is not a huge amount of money but that will help pay for the VPS I rent at OVH to host my projects at https://muscle.apdu.fr/. They are mostly pcsc-lite and libccid.

Conclusion

I have 2 active options to send me money:
Feel free to use whatever to prefer.

Friday, May 15, 2020

New PyKCS11 1.5.8 available

I just released a new version of PyKCS11, a Python wrapper above the PKCS#11 API.
See "PyKCS11 introduction" or "PyKCS11’s documentation".

The project is registered at Pypi: https://pypi.org/project/PyKCS11/

Changes:

1.5.8 - May 2020, Ludovic Rousseau
  • CKA_ALWAYS_AUTHENTICATE is boolean
  • CKM_VENDOR_DEFINED_...
    • Fix name: use CKM_ instead of CKR_ prefix
    • Use an explicit hex prefix: CKM_VENDOR_DEFINED_0x45
  • Add missing CKM_*, CKA_*, CKF_*, CKD_*, CKK_*, CKN_*, CKO_*, CKR_* from PKCS#11 v3.0
  • fix test_asymetric.py for RSA_PSS_Mechanism

Friday, May 8, 2020

Your PC/SC application 200% faster

Dana Keeler reported an issue on pcsc-lite at "SYS_USleep in SCardEndTransaction in winscard_clnt.c causing slowness in Firefox".

The problem was initially reported on Firefox "firefox is very slow and crashes because of p11-kit-proxy.so" and also on p11-kit "firefox is very slow and crashes because of p11-kit-proxy.so".

The issue

The pcsc-lite performance issue comes from this piece of code in SCardEndTransaction:

 /*
  * This helps prevent starvation
  */
 randnum = SYS_RandomInt(1000, 10000);
 (void)SYS_USleep(randnum);

For each call to SCardEndTransaction() you will get a delay between 1 ms and 11 ms. So on average you get 6 ms of delay each time.

Performance measures

I wrote a Python program to get the duration of 100 calls to SCardEndTransaction():

#! /usr/bin/env python3

from smartcard.scard import *
from smartcard.pcsc.PCSCExceptions import *
from time import time

hresult, hcontext = SCardEstablishContext(SCARD_SCOPE_USER)
if hresult != SCARD_S_SUCCESS:
    raise EstablishContextException(hresult)

hresult, readers = SCardListReaders(hcontext, [])
if hresult != SCARD_S_SUCCESS:
    raise ListReadersException(hresult)
print('PC/SC Readers:', readers)
reader = readers[0]
print("Using reader:", reader)

hresult, hcard, dwActiveProtocol = SCardConnect(hcontext, reader, SCARD_SHARE_SHARED, SCARD_PROTOCOL_ANY)
if hresult != SCARD_S_SUCCESS:
    raise BaseSCardException(hresult)

nb = 100
total = 0
for i in range(nb):
    hresult = SCardBeginTransaction(hcard)
    if hresult != SCARD_S_SUCCESS:
        raise BaseSCardException(hresult)

    before = time()
    hresult = SCardEndTransaction(hcard, SCARD_LEAVE_CARD)
    if hresult != SCARD_S_SUCCESS:
        raise BaseSCardException(hresult)
    after = time()
    delta = after - before
    print(delta)
    total += delta
print("total: {} ms".format(total * 1000))
print("average: {} ms".format(total * 1000 / nb))

hresult = SCardDisconnect(hcard, SCARD_LEAVE_CARD)
if hresult != SCARD_S_SUCCESS:
    raise BaseSCardException(hresult)

hresult = SCardReleaseContext(hcontext)
if hresult != SCARD_S_SUCCESS:
    raise ReleaseContextException(hresult)

Results


As expected the random delay is of 6 ms (or 0.006 second) on average.

The history

I don't know why this delay is needed. I went into the previous versions of the source code file PCSC/src/winscard_clnt.c but the oldest version (from March 2002, 18 years ago) already has this delay.

The code was written David Corcoran, the initial author of pcsc-lite. It was my very early days in the pcsc-lite project at that time.

The solution

I don't see any good reasons to have a delay here. I guess it was to solve a problem with the communication between pcscd and libpcsclite at that time. The communication mechanism has been redesigned in version 1.6.0 (May 2010) and the delay should now be useless and even problematic as we saw.

My solution is then to remove the problematic code. The was done in this commit.

The results

I used again my Python program to measure the performances of SCardEndTransaction(). I now have:

The mean delay is 9.5x10-6 s so 9 ยตs or 0.009 ms or 0.000009 second. The speedup factor is huge: x647.
The function is now 600 times faster than before.

You can note a high decrease on the 4 first values. My guess is that it is an effect of CPU memory caches in action. I have not investigated this point. This is left as an exercise for my readers.

Macro benchmark

It is nice to have a huge improvement in one PC/SC function but does that help real applications?

For a real smart card application I used OpenSC with a standard command: list all the objects of a smart card.
I used a very simple shell script:

#!/bin/bash

for i in {1..100}
do
 pkcs11-tool --list-objects
done

Slow smart card

First I used a very old smart card I have in my collection: a Gemplus GPK 8000 card.
Since the card is very old and slow the shell script will do 10 rounds instead of 100.

whattime (s)
Before23.34
After22.84

Speedup: x1.02 or 2%

The gain is very limited. This is because the card is so slow that the benefit from the new SCardEndTransaction() is negligible.

Fast smart card

I then used a much faster smart card: a Yubikey 5 from Yubico. It is not a real smart card but a token with a CCID interface and a chip that understand APDU commands.
Since the device is fast I used 100 rounds of pkcs11-tool.

whattime (s)
Before9.85
After3.02

Speedup: x3.26 or 226%

This time the gain is highly visible. The pkcs11-tool command is now 3 times faster.

Results

You will get a high acceleration with fast smart card.
I was able to get a full execution of pkcs11-tool 3 (three) times faster than before the change.

I was really impressed by this result. For almost 10 years pcsc-lite was slow and could be improved by just removing 2 lines of code.

Potential regression?

I don't think this change will create a regression and will break existing code.

I made a beta version of pcsc-lite including the change available at http://ludovic.rousseau.free.fr/softwares/pcsc-lite/pcsc-lite-1.8.26-05d48e5.tar.bz2.
Please test this version with your PC/SC application. In case of a problem with PC/SC transactions then open an issue at Salsa https://salsa.debian.org/rousseau/PCSC/-/issues or github https://github.com/LudovicRousseau/PCSC/issues.

I plan to wait a few weeks to get potential feedback before I make a new release of pcsc-lite.

Conclusion

pcsc-lite is now much faster.

This happened because someone complained that Firefox was slow and someone at Mozilla investigated the issue to find a problem in pcsc-lite.

I am very happy to use Free Software programs where it is possible and easy to find problems in software you use.

Friday, May 1, 2020

PCSC sample in Free Pascal (Lazarus)

Here is a new PCSC sample in Free Pascal language I promised in PC/SC sample in different languages.

Lazarus

From Lazarus web site:
What is Lazarus?
Lazarus is a Delphi compatible cross-platform IDE for Rapid Application Development. It has variety of components ready for use and a graphical form designer to easily create complex graphical user interfaces.

Infintuary Pascal PC/SC Sample

I found a Pascal PC/SC Sample code for PCSC with Lazarus at http://infintuary.org/stpcsc.php.

This program is also available at https://github.com/ccy/pcsc in a more recent version. Unfortunately the more recent version has Windows specific code and can't be used on GNU/Linux. I reported the issue at Fix build on GNU/Linux: CheckOSError() is for Windows.

License

The license is custom but could be enough. From http://infintuary.org/stpcsc.php
The Pascal PC/SC Sample is freeware and can be used for any purpose.
By downloading the Pascal PC/SC Sample you agree to the terms of use.
The terms of use is different from the license available on the github project. For example the terms of use does not explicitly allow modification of the source code, and the use is allowed only for legal purpose. In general it is a bad idea to re-invent a new license text.

Installation

You copy the files MD_Events.pas, MD_PCSC.pas, MD_PCSCDef.pas, MD_PCSCRaw.pas and MD_Tools.pas.

Source code


unit main;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls,
  MD_PCSCRaw, MD_PCSCDef, MD_Tools;

type

  { TForm1 }

  TForm1 = class(TForm)
    Button1: TButton;
    Memo1: TMemo;
    procedure Button1Click(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure AddLogMemo(Msg: string);
  private
    FPCSCRaw: TPCSCRaw;

  public

  end;

var
  Form1: TForm1;

implementation

{$R *.lfm}

{ TForm1 }

procedure TForm1.Button1Click(Sender: TObject);
var
  PCSCResult: Dword;
  hContext: THandle;
  SizeReaders: LongWord;
  pReaders: PChar;
  hCard: LongInt;
  dwActiveProtocol: Cardinal;
  pioSendPCI, pioRecvPCI: pSCardIORequest;
  inBuffer : TBytes;
  outBuffer: TBytes;
  outSize: Cardinal;
  i: Integer;
  outString: String;
begin
    FPCSCRaw := TPCSCRaw.Create;
    FPCSCRaw.Initialize;

    // Establish context
    PCSCResult := FPCSCRaw.SCardEstablishContext(SCARD_SCOPE_SYSTEM, nil, nil, hContext);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardEstablishContext succeeded.')
      else AddLogMemo('SCardEstablishContext failed: ' + PCSCErrorToString(PCSCResult));

    // List readers
    PCSCResult := FPCSCRaw.SCardListReaders(hContext, nil, nil, SizeReaders);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardListReaders succeeded.')
      else AddLogMemo('SCardListReaders failed: ' + PCSCErrorToString(PCSCResult));

    GetMem(pReaders, SizeReaders);
    PCSCResult := FPCSCRaw.SCardListReaders(hContext, nil, pReaders, SizeReaders);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardListReaders succeeded.')
      else AddLogMemo('SCardListReaders failed: ' + PCSCErrorToString(PCSCResult));

    // Use the first reader
    AddLogMemo('Using: ' + pReaders);

    // Connect to the card
    hCard := -1;
    dwActiveProtocol := 0;
    PCSCResult := FPCSCRaw.SCardConnect(hContext, pReaders, SCARD_SHARE_SHARED,
      SCARD_PROTOCOL_Tx, hCard, dwActiveProtocol);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardConnect succeeded.')
      else AddLogMemo('SCardConnect failed: ' + PCSCErrorToString(PCSCResult));

    // Send Select Applet command
    pioRecvPCI := nil;
    if dwActiveProtocol = SCARD_PROTOCOL_T0
      then pioSendPCI := @SCARDPCIT0
      else pioSendPCI := @SCARDPCIT1;

    inBuffer := HexStringToBuffer('00 A4 04 00 0A A0 00 00 00 62 03 01 0C 06 01');
    outSize := 258;
    SetLength(outBuffer, outSize);
    PCSCResult := FPCSCRaw.SCardTransmit(hCard, pioSendPCI, Pointer(inBuffer),
      length(inBuffer), pioRecvPCI, Pointer(outBuffer), outSize);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardTransmit succeeded.')
      else AddLogMemo('SCardTransmit failed: ' + PCSCErrorToString(PCSCResult));

    SetLength(outBuffer, outSize);
    AddLogMemo('Received (' + IntToStr(outSize) + ' bytes): ' + BufferToHexString(outBuffer));

    // Send test command
    inBuffer := HexStringToBuffer('00 00 00 00');
    outSize := 258;
    SetLength(outBuffer, outSize);
    PCSCResult := FPCSCRaw.SCardTransmit(hCard, pioSendPCI, Pointer(inBuffer),
      length(inBuffer), pioRecvPCI, Pointer(outBuffer), outSize);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardTransmit succeeded.')
      else AddLogMemo('SCardTransmit failed: ' + PCSCErrorToString(PCSCResult));

    SetLength(outBuffer, outSize);
    AddLogMemo('Received (' + IntToStr(outSize) + ' bytes): ' + BufferToHexString(outBuffer));

    outString := '';
    for i := 0 to outSize -3 do
      outString := outString + chr(outBuffer[i]);
    AddLogMemo(outString);

    // Disconnect
    PCSCResult := FPCSCRaw.SCardDisconnect(hCard, SCARD_LEAVE_CARD);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardDisconnect succeeded.')
      else AddLogMemo('SCardDisconnect failed: ' + PCSCErrorToString(PCSCResult));

    // Release context
    PCSCResult := FPCSCRaw.SCardReleaseContext(hContext);
    if PCSCResult = SCARD_S_SUCCESS
      then AddLogMemo('SCardReleaseContext succeeded.')
      else AddLogMemo('SCardReleaseContext failed: ' + PCSCErrorToString(PCSCResult));
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
  Memo1.Clear;
end;

procedure TForm1.AddLogMemo(Msg: string);
begin
  Memo1.Lines.Add(Msg);
end;

end.


Remarks

You will have to create a Form with a TMemo widget and a TButton widget. This part of the solution is not described here.

You click on the button and the program executes.

Output



Comments

High level API

I used the low level API provided in file MD_PCSCRaw.pas. These are direct equivalent of the WinSCard API. But a higher level API is also available and is provided in the file MD_PCSC.pas.

Unfortunately I was not able to find a way to get the list of connected readers with the high level API. Maybe I missed something (I am very new to Free Pascal). I reported the issue at make TPCSC.GetPCSCReaderList() a public method.

64-bits and GNU/Linux issue

A much more important issue is that this wrapper does not work on a 64-bits GNU/Linux system. I had to install a Debian system on a i386 CPU to write and test my code.

The problem is that the wrapper uses a Free Pascal type THandle to store the SCARDCONTEXT and SCARDHANDLE types. The problem is that a THandle in 32-bits on a 64-bits CPU but SCARDCONTEXT and SCARDHANDLE are 64-bits types on a 64-bits CPU using GNU/Linux.

The problem comes from the definition of long (DWORD) on a 64-bits CPU. On GNU/Linux a long is 64-bits. But on Windows a long is 32-bits only. If you want to have a 64-bits variable on Windows you need to use the long long type. This is because Linux is LP64 and Windows is LLP64. See 64-bit data models for more details.

The wrapper should work on macOS even on a 64-bits CPU. This is because Apple does not use any DWORD in the WinSCard API but int32_t instead.

I reported the problem at Using THandle for SCARDCONTEXT and SCARDHANDLE is wrong on GNU/Linux.

Windows only?

As I already wrote in the introduction the, more recent, code available at https://github.com/ccy/pcsc is for Windows only. Some work is needed to make it available (again) for macOS and GNU/Linux.

Conclusion

If you use the Free Pascal language and you use Windows then this wrapper can help you.
If your system is not Windows or you want portability on diffrent systems then this wrapper may not be a good choice.