C native data types in Lisp

5 Dec 2016 - 20:17

This is an example of how to use Common Lisp for the wrong purposes, although this shows that it's closer to a systems language than most of the other high-level modern languages.

Since Lisp has its own implementation-dependent data types, as most high-level languages do, any interaction with the world outside of the Lisp image, such as binary data import/export or code involving system calls or external C functions, is expected to be painful to some degree.

Surprisingly, most serious implementations have a nice enough FFI system to make C-native data definitions and alien function calls very easy, but I'm more concerned with data processing, which means that at some point I'd want to send those native "variables" to some place other than memory.

I am strongly against the use of raw binary formats but that's how the world works. unfortunately. I know that if your application interface depends on tightly-packed C structs you're already screwed, but still, I'd like to be able to do that easily in Lisp like you do in, for example, Perl (pack/unpack). Also, the recommended way to do this in Lisp is to define your own Lisp-to-C data mapping functions manually, which is really not practical. SBCL already knows how to do the mappings, but these features are rather obscure and scarcely documented.

The common link I'm using to bridge the alien C datatypes in SBCL -comprehensive and well documented, by the way- and real Lisp variables is the byte array (an octet is an octet everywhere), so the key to this code is how to collect any C datatype into a byte array that you can later dump into a file or whatever.

;;;; ==== Tested on SBCL 1.3.9 ====
;;; Define a C struct type
(define-alien-type nil
    (struct foo
            (a char)
            ;; There should be 3 bytes of padding here
            (b int)))

;;; This returns the size of an arbitrary alien object
(defun alien-obj-size (alien-obj)
  (- (sb-sys:sap-int (alien-sap (addr (deref alien-obj 1))))
     (sb-sys:sap-int (alien-sap (addr (deref alien-obj 0))))))

;;; This is equivalent to a naive C memset (byte per byte)
(defun memset (alien-obj value)
  (dotimes (i (alien-obj-size alien-obj))
    (setf (sb-sys:sap-ref-8 (alien-sap alien-obj) i) value)))

;;; Create an alien variable of type (struct foo) and populate it.
;;; It may content garbage, and setting the slots individually write only each
;;; slot's width, so the padding between them may still be garbage. memset it
;;; with zeros first
(defparameter *a* (make-alien (struct foo)))
(memset *a* 0)
(setf (slot *a* 'a) 1)
(setf (slot *a* 'b) 255)

;;; Get the object's SAP directly as a byte array
(defun coerce-alien (alien-obj)
  (sb-sys:sap-ref-octets (alien-sap alien-obj)
                          (alien-obj-size alien-obj)))

;;; Bulk-write test
    (str "testfile"
         :direction :output
         :if-exists :supersede
         :element-type '(unsigned-byte 8))
  (write-sequence (coerce-alien *a*) str))

Software - Lisp - Tutorials