Ctypes pass OCaml bytes to C functions?

Learning
,
1

Hello,

I’m trying to call a C function with roughly the following signature

int calc_stuff (const uint8_t* input, uint8_t* out);

It does some computation on the C side and writes to out.

I’m currently using Ctypes on the OCaml side, and have made roughly what follows

let calc_stuff =
  foreign "calc_stuff" (ptr uint8_t @-> ptr uint8_t -> returning int)

let calc (input: string) (out : bytes) : int =
  calc_stuff
    (coerce string (ptr uint8_t) input)
    ( ??? )

The issue is, I am not really sure what to put in the ( ??? ) part.

I have tried (coerce string (ptr uint8_t) (Bytes.unsafe_to_string out)), but this will not modify out, since (as far as I understand) strings are copied to outside of OCaml GC, and thus the shared ownership no longer exists.

My questions are

  1. Is there a way to pass OCaml bytes directly to C functions?
  2. Is there a way to pass OCaml string directly to C functions such that no copying is done?

Thanks in advance!

2

You need to allocate CArrays or Bigbuffers and use them both for the input string and the mutable buffer, otherwise you will get in big troubles with the Gc. Look, for example, at: https://github.com/simonjbeaumont/ocaml-pci/pull/20 (if you use the latest ctypes this is even simpler as you can do CArray.of_string)

Some brief information are also here: https://github.com/ocamllabs/ocaml-ctypes/wiki/FAQ#strings

I have been planning on writing a blog post on that and other issues that we have ecountered in dealing with strings and ctypes bindings… I should do it sooner rather than later

2 Likes
3

If you really want to reduce the number of copies and allocations, you are probably better off writing directly c stubs by hand, have a look at the “Interfacing with C” section of the ocaml manual. There you have String* and Bytes* accessors to read and write (directly I believe) in the ocaml strings. But double check the manual and read it carefully before believing me

1 Like
4

Thanks very much for the reply!

I think I will migrate to using Bigarray to avoid things getting messy as multicore lands.

5

@mseri Do you happen to also know if Bigstring.of_string and Bigstring.to_string are doing anything special for quickly constructing the internal Bigarray and string or is it just copying?

I intend to use

(int, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t

as it makes calculations easier at places, but Bigstring is

(char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t

and I cannot seem to find a function for converting between the two.

Just trying to figure out if there is any performance difference between Bigstring.of_string and just copying it into the (int, _, _) Bigarray.Array1.t.

Thanks!

6

I don’t have any experience with them, sorry. If you don’t get an answer here you can tryin the ocaml-ctypes mailing list

1 Like
7

of_string copies the data, since the memory representation of a string and a Bigarray.t are not the same.

However, since the memory representation of Bigarray is compatible with that of C arrays (using the correct layout), you can create a Bigarray from a C pointer using caml_ba_alloc. This will not incur any copying.

Unfortunately, I don’t know of any mechanism to convert a Bigarray from one kind to another, so I don’t know if you’d be able to use Bigstring, but if you create the Bigarray with the correct flags from the C side you can use the Int8_unsigned kind on the OCaml side.

2 Likes