Skip to content

std::csv

Reading and writing CSV, as RFC 4180 describes it: records of fields separated by a delimiter, one record a line, a field that holds the delimiter, a quote or a line break written in quotes with each quote doubled.

import std::csv

rows = csv.decode_rows("name,age\nAda,36\n")?

ages = rows |> Iter.map(fn row -> csv.Row.get(row, "age")) |> Maybe.values()

csv.decode reads text whole into records, and csv.decode_rows reads its first record as a header and each one after as a Row, whose fields are found by column name. csv.records and csv.rows read a Reader a record at a time, for a file too large to hold. Options sets the delimiter, a comment character, trimming of leading space, lenient quotes and the field count each record must have.

csv.encode and csv.write write records, quoting each field that needs it. WriteOptions.escape_formulas keeps a spreadsheet from running a field as a formula.

Reading is Go’s encoding/csv. A UTF-8 byte order mark at the start is dropped, a \r\n is read as \n, inside a quoted field too, and an empty line is skipped. A failure names its line and column, the column counted in code points from 1. The text must be UTF-8.

Import with import std::csv.

subtype Delimiter Codepoint when fn c -> c != '"' and c != '\r' and c != '\n' and c != '\u{0}' and Codepoint.to_int(c) != 0xFFFD

A code point that can separate fields or start a comment: any but a quote, \r, \n, U+FFFD and U+0000, which a reader could not tell apart from the text around it. Options and WriteOptions take one, so Options{delimiter: '"'} is a compile error, and a delimiter read when the program runs goes through Delimiter.try_from.

subtype: Delimiter is a subtype of Codepoint. A literal where a Delimiter is expected is checked when the program is compiled; Delimiter.try_from(...) checks a value when it runs.

tab: Delimiter = '\t'
assert decode("a\tb", Options{delimiter: tab}) == Ok([["a", "b"]])
assert Delimiter.try_from('"') == None
assert Delimiter.try_from('\n') == None
assert Delimiter.try_from(';') == Some(';')
fn try_from(value: Codepoint) -> Maybe<Delimiter>

type function on Delimiter

Some of the value when it satisfies Delimiter’s rules, None when it does not.

enum FieldCount
    SameAsFirst
    Any
    Exactly Int
end

How many fields each record must have.

Variants

  • SameAsFirst

    As many as the first record. A record with another count is a Syntax error.

  • Any

    Any number: records may differ.

  • Exactly

    Exactly this many.

impl Equatable for FieldCount

fn equal(a: FieldCount, b: FieldCount) -> Bool

impl Equatable.equal

impl Debug for FieldCount

fn inspect(value: FieldCount) -> String

impl Debug.inspect

struct Options
    delimiter: Delimiter = ','
    comment: Maybe<Delimiter> = None
    trim_leading_space: Bool = False
    lazy_quotes: Bool = False
    fields: FieldCount = FieldCount.SameAsFirst
end

How decode, records and the other readers read CSV.

assert decode("a;b\n# note\nc;d", Options{delimiter: ';', comment: Some('#')})
    == Ok([["a", "b"], ["c", "d"]])

assert decode("a, b", Options{trim_leading_space: True}) == Ok([["a", "b"]])
assert decode("a\"b,c", Options{lazy_quotes: True}) == Ok([["a\"b", "c"]])
assert decode("a\nb,c", Options{fields: .Any}) == Ok([["a"], ["b", "c"]])

impl Debug for Options

fn inspect(value: Options) -> String

impl Debug.inspect

enum Error
    Syntax { message: String, line: Int, column: Int }
    Io { error: io.Error }
    InvalidOptions { message: String }
end

Why CSV could not be read.

Variants

  • Syntax

    The text is not CSV, or a record has the wrong field count, at line and column, both from 1, the column in code points.

  • Io

    The Reader failed.

  • InvalidOptions

    The Options cannot be used: the delimiter and the comment character are the same, or the field count is negative.

assert decode("a,\"b") |> Result.map_err(&Display.to_string(&1)) == Err(
    "line 1, column 5: a quoted field is not closed, or has a quote in it that is not doubled"
)

impl Display for Error

fn to_string(error: Error) -> String

impl Display.to_string

impl Equatable for Error

fn equal(a: Error, b: Error) -> Bool

impl Equatable.equal

impl Debug for Error

fn inspect(value: Error) -> String

impl Debug.inspect

struct Row
    header: List<String>
    fields: List<String>
    line: Int
end

A record read under a header: the header’s columns, the record’s fields in the same order, and the line the record starts on.

rows = decode_rows("id,name\n7,Ada")!

assert rows == [
    Row{header: ["id", "name"], fields: ["7", "Ada"], line: 2},
]
fn get(row: Row, column: String) -> Maybe<String>

type function on Row

The field in the column named column, the first such column when the header names it twice, or None when the header does not name it.

row = Row{header: ["id", "name"], fields: ["7", "Ada"], line: 2}
assert Row.get(row, "name") == Some("Ada")
assert Row.get(row, "email") == None
fn to_map(row: Row) -> Map<String, String>

type function on Row

The fields by column name. When the header names a column twice, the first one’s field is kept.

row = Row{header: ["a", "b", "a"], fields: ["1", "2", "3"], line: 1}
assert Row.to_map(row) == {"a" => "1", "b" => "2"}

impl Equatable for Row

fn equal(a: Row, b: Row) -> Bool

impl Equatable.equal

impl Debug for Row

fn inspect(value: Row) -> String

impl Debug.inspect

fn decode(text: String, options: Options = Options{}) -> Result<List<List<String>>, Error>

Every record in text, each a list of its fields.

assert decode("a,b\r\n\"c,\"\"d\"\"\",\"e\nf\"\n") == Ok([["a", "b"], ["c,\"d\"", "e\nf"]])

assert decode("\u{feff}x,y") == Ok([["x", "y"]])
assert decode("") == Ok([])

assert decode("a,b\nc") == Err(Error.Syntax{
        message: "the record has 1 fields, not 2",
        line:    2,
        column:  1,
    })

assert decode("é,\"x\"y") == Err(Error.Syntax{
        message: "a quoted field is not closed, or has a quote in it that is not doubled",
        line:    1,
        column:  5,
    })
fn decode_bytes(data: Bytes, options: Options = Options{}) -> Result<List<List<String>>, Error>

Every record in data, which must be UTF-8.

assert decode_bytes(String.to_bytes("a,b")) == Ok([["a", "b"]])

assert decode_bytes(Bytes.from_list([97, 44, 255])) == Err(Error.Syntax{
        message: "the text is not UTF-8",
        line:    1,
        column:  3,
    })
fn decode_rows(text: String, options: Options = Options{}) -> Result<List<Row>, Error>

The records of text after its first, each a Row under the header the first record is. Text with no records has no rows.

rows = decode_rows("name,age\nAda,36\nAlan,41\n")!
assert rows |> Iter.map(fn row -> Row.get(row, "age")) |> Iter.to_list() == [Some("36"), Some("41")]
assert decode_rows("name,age\n") == Ok([])
fn records(reader: Reader, options: Options = Options{}) -> Iter<Result<List<String>, Error>>

Each record left in reader, read as the iterator runs, so a large file is never held whole. A failure is an item, and the last one. Like Reader.lines, it runs once: a second run carries on from where the stream is.

r = Reader.from_text("a,b\nc,d\n")
assert Result.collect(records(r)) == Ok([["a", "b"], ["c", "d"]])

bad = records(Reader.from_text("a\n\"b"))

assert Iter.to_list(bad) == [Ok(["a"]), Err(Error.Syntax{
        message: "a quoted field is not closed, or has a quote in it that is not doubled",
        line:    2,
        column:  3,
    })]
fn rows(reader: Reader, options: Options = Options{}) -> Iter<Result<Row, Error>>

Each record left in reader after the first, as a Row under the header the first record is, read as the iterator runs. It runs once, as records does: the header is the first record the first run reads.

r = Reader.from_text("id,name\n1,Ada\n2,Alan\n")

names =
    rows(r)
    |> Iter.map(fn row -> Result.map(row, fn row -> Row.get(row, "name")))
    |> Result.collect()

assert names == Ok([Some("Ada"), Some("Alan")])
struct WriteOptions
    delimiter: Delimiter = ','
    crlf: Bool = False
    escape_formulas: Bool = False
end

How encode and write write CSV.

assert encode([["a", "b"]], WriteOptions{delimiter: '\t', crlf: True}) == "a\tb\r\n"
assert encode([["=SUM(A1:A9)", "-3"]], WriteOptions{escape_formulas: True}) == "'=SUM(A1:A9),'-3\n"

impl Debug for WriteOptions

fn inspect(value: WriteOptions) -> String

impl Debug.inspect

fn encode(records: Iter<List<String>>, options: WriteOptions = WriteOptions{}) -> String

records as CSV text, each record a line. A field is quoted when it holds the delimiter, a quote, a line break or white space at its start, and each quote in it is doubled. A record of no fields is an empty line, which a reader skips; one empty field is written "".

assert encode([["name", "note"], ["Ada", "said \"hi\", left"], ["Alan", "two\nlines"]])
    == "name,note\nAda,\"said \"\"hi\"\", left\"\nAlan,\"two\nlines\"\n"

records = [["a b", " c", ""], [""]]
assert decode(encode(records), Options{fields: .Any}) == Ok(records)
fn write(w: Writer, records: Iter<List<String>>, options: WriteOptions = WriteOptions{}) -> Result<Unit, io.Error>

Writes records to w as encode writes them, a record at a time, and stops at the first write that fails.

(written, data) = Writer.collect(fn w -> write(w, [["a", "b,c"]]))
assert written == Ok(Unit)
assert Bytes.to_string(data) == Ok("a,\"b,c\"\n")