C / CAPSTONE PROJECTS
Project: a CSV parser with quoted-field support
Write a C CSV parser that treats commas and newlines inside quoted fields as data, collapses "" to one quote, and handles multi-line records.
What you will learn
- Split a CSV record with a quote-state machine instead of strtok or strchr
- Collapse a doubled quote inside a quoted field into one literal quote
- Detect records that span lines by carrying quote state across fgets calls
- Unescape in place safely: a decoded field is never longer than its raw text
Understanding Project: a CSV parser with quoted-field support
CSV looks like a format you can handle with strtok, and that impression survives until the first field that contains a comma. Quoting makes the delimiters context-dependent: a comma separates fields only when the parser is outside a quoted field, and inside one it is ordinary data. So every correct CSV reader is a small state machine that remembers whether the current character sits inside quotes, and no stateless splitting function can carry that memory for you.
Three rules do all the work. A field is quoted when its first character is a double quote; inside a quoted field two adjacent double quotes stand for one literal quote; and inside a quoted field, commas and newlines are data. The third rule has a consequence beginners miss: a record is not the same thing as a line, so a single fgets call can hand you half a record. You can detect that by flipping a flag on every quote byte, since an escaped pair flips it twice and cancels out, leaving the flag set exactly when the record is unfinished.
Decoding a field never makes it longer: removing the surrounding quotes and collapsing a doubled quote into one only deletes bytes. Because the write index can therefore never overtake the read index, you may unescape in place in a mutable copy of the record, writing a NUL over each delimiter and keeping pointers to the field starts. The parser below copies into a caller-supplied buffer instead, and returns the byte that ended the field, comma or newline or NUL, because that one return value is what tells a row-oriented loop when the record is over.
The interface matters as much as the state machine: a function that fills one field and reports its terminator composes into any row loop, while a function that returns an array of fields forces an allocation policy on every caller.
<stdio.h>
/* Copy the next field of a CSV record from *cur into out.
Returns the byte that ended the field: ',', '\n' or '\0'. */
static int next_field(const char **cur, char *out, size_t cap)
{
const char *p = *cur;
size_t n = 0;
int quoted = (*p == '"');
if (quoted)
p++;
while (*p != '\0') {
if (quoted) {
if (*p == '"') {
if (p[1] == '"') { /* doubled quote -> one quote */
if (n + 1 < cap)
out[n++] = '"';
p += 2;
continue;
}
quoted = 0; /* closing quote */
p++;
continue;
}
} else if (*p == ',' || *p == '\n') {
break; /* delimiter: we are outside quotes */
}
if (n + 1 < cap)
out[n++] = *p;
p++;
}
out[n] = '\0';
*cur = (*p == '\0') ? p : p + 1;
return *p;
}
int main(void)
{
const char *data =
"name,quote,city\n"
"Ada,\"She said \"\"yes\"\", loudly\",London\n"
"Bob,\"line one\nline two\",\n";
const char *cur = data;
char field[64];
int row = 0, col = 0;
while (*cur != '\0') {
int end = next_field(&cur, field, sizeof field);
printf("row %d col %d: [%s]\n", row, col, field);
if (end == ',') {
col++;
} else {
row++;
col = 0;
}
}
return 0;
}
A comma or newline is a delimiter only while the parser is outside a quoted field, so CSV must be parsed with carried quote state rather than split.
Worked examples
Why strtok gets the field count wrong
The same record counted by a quote-aware scan and by strtok, showing where the columns shift.
<stdio.h>
<string.h>
/* Fields = commas seen outside quotes, plus one. */
static int quoted_count(const char *s)
{
int n = 1, inq = 0;
for (; *s != '\0'; s++) {
if (*s == '"')
inq = !inq;
else if (*s == ',' && !inq)
n++;
}
return n;
}
/* strtok knows nothing about quotes; it cuts at every comma. */
static int naive_count(char *s)
{
char *tok = strtok(s, ",");
int n = 0;
while (tok != NULL) {
printf(" naive[%d] = %s\n", n++, tok);
tok = strtok(NULL, ",");
}
return n;
}
int main(void)
{
const char *orig = "1,\"Smith, John\",42";
char line[] = "1,\"Smith, John\",42"; /* strtok writes into its input */
int aware = quoted_count(orig);
int naive;
printf("input: %s\n", orig);
printf("quote-aware fields: %d\n", aware);
naive = naive_count(line);
printf("strtok fields: %d\n", naive);
return 0;
}
Example explained
Line 1quoted_count starts n at 1 because n delimiters outside quotes produce n+1 fields.
Line 2The inq flag is flipped by every quote byte, so the comma inside the quoted name never reaches the counting branch.
Line 3strtok has no such flag, so it cuts inside the name: column 1 becomes a fragment and every later column is shifted by one.
Line 4naive_count is handed a separate char array because strtok writes NUL bytes into its argument, which a string literal must never receive.
A record that spans three input lines
Buffering lines until the quote count is even, which is how an embedded newline is preserved.
<stdio.h>
<string.h>
/* Flip the state on every quote byte; a doubled quote flips twice. */
static int track_quotes(const char *s, int inq)
{
for (; *s != '\0'; s++)
if (*s == '"')
inq = !inq;
return inq;
}
int main(void)
{
const char *lines[] = {
"id,note\n",
"7,\"first half\n",
"second half\"\n"
};
char record[128] = ""; /* small on purpose; real code must bound this */
int inq = 0;
size_t i;
for (i = 0; i < sizeof lines / sizeof lines[0]; i++) {
strcat(record, lines[i]);
inq = track_quotes(lines[i], inq);
if (inq) {
printf("held: quote still open\n");
continue;
}
printf("record: %s", record);
record[0] = '\0';
}
return 0;
}
Example explained
Line 1track_quotes takes the previous state and returns the new one, so quote state survives from one line to the next instead of being reset.
Line 2After the second line one quote is still open, so the loop only appends to record and prints held rather than parsing.
Line 3The third line closes the quote, the state returns to 0, and the record that is printed contains a real newline inside its second field.
Line 4A doubled quote flips the flag twice, so this even/odd test stays correct for fields that contain escaped quotes.
Important notes
This parser is lenient about malformed input: bytes after a closing quote, as in "ab"cd, are simply appended to the field. Strict readers reject that, so pick a behaviour and document it.
Windows-written files end records with CR LF. Strip the CR that sits immediately before the record's newline, but leave any CR inside a quoted field alone, because there it is data.
Common mistakes
Splitting with strtok or strchr: a quoted field containing a comma becomes two fields, every later column shifts by one, and the row is silently corrupted instead of rejected.
Treating a doubled quote as an escape to skip without emitting anything, so a field holding say "hi" decodes to say hi and the quote characters vanish from the data.
Assuming one fgets line is one record: a field with an embedded newline is cut in half and the tail is parsed as a fresh unquoted row whose commas split data.
Try it yourself
Change, predict, then run
Add an out-parameter to next_field that reports whether the field was quoted, then print NULL for an empty unquoted field and the empty string for a field written as two quotes, so the two cases stop looking identical.
Open the C workspaceCheck your understanding
A parser reads one line per fgets call and tracks quote state while splitting, but resets that state to "outside quotes" at the start of every line. What breaks?
- Doubled quotes decode one character too early, so each escaped quote loses a byte.
- A field containing a newline is split into two records, and the text after the newline is parsed with its commas treated as delimiters.
- Nothing breaks; quote state cannot legally cross a newline, so resetting per line is required.
- The first field of each record is dropped whenever the previous line ended inside a quoted field.
Show answer
Quoted fields are allowed to contain newlines, so one record can occupy several lines and the quote state must be carried across fgets calls until the count is even. Option 3 is tempting because most CSV files really do put one record per line, but the quoting rules explicitly permit embedded newlines, and those are exactly the files where a per-line reset turns one row into two malformed ones.