JAVA / FILES, INPUT/OUTPUT AND SERIALIZATION
Byte streams versus character streams
Choose between Java's byte streams and character streams for a given payload, bridge them with an explicit charset, and predict where bytes and chars diverge.
What you will learn
- Pick InputStream/OutputStream for binary formats, Reader/Writer for text you interpret.
- Bridge them with InputStreamReader/OutputStreamWriter and an explicit Charset.
- Explain why String.length() counts UTF-16 chars, not bytes or characters.
- Spot irreversible U+FFFD damage when binary bytes are decoded as text.
Understanding Byte streams versus character streams
Java has two parallel I/O hierarchies because a disk, a socket and a pipe only ever hold bytes, while your program usually wants characters. InputStream and OutputStream work in 8-bit units: read() hands back an int from 0 to 255, or -1 at end of input. Reader and Writer work in UTF-16 code units: read() hands back an int from 0 to 65535, or -1. Nothing in the file itself tells you which family is right; that is a decision about what the bytes are supposed to mean.
Characters do not exist on disk, they are produced by decoding, and decoding requires a charset. InputStreamReader and OutputStreamWriter are the bridge where that happens, and they are the only classes in the picture that hold a Charset. Decoding is stateful because one character can span several bytes, so the bridge keeps partial sequences between calls and may pull in more bytes than you asked for. That is why you cannot reliably decode a stream one byte at a time yourself, and why alternating reads between a Reader and its underlying InputStream loses data.
The two families fail in opposite directions. Push a PNG through a Reader and every byte sequence that is not valid UTF-8 becomes U+FFFD, the replacement character, which is a one-way trip: the original byte is gone and re-encoding emits three bytes where there was one. Push text through a byte stream while assuming one char is one byte and any non-ASCII content gets truncated or split mid-character. The working rule is that a human or a text specification defining the content means character streams with an explicit charset, while a binary format defining it means byte streams with String kept out of the path entirely.
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
public class ByteVsChar {
public static void main(String[] args) throws IOException {
String text = "caf\u00e9"; // c a f e-acute, escaped so the source file stays ASCII
byte[] utf8 = text.getBytes(StandardCharsets.UTF_8);
System.out.println("chars in string: " + text.length());
System.out.println("utf-8 bytes: " + utf8.length);
// A byte stream moves those bytes and never asks what they mean.
ByteArrayOutputStream sink = new ByteArrayOutputStream();
sink.write(utf8);
System.out.println("bytes copied: " + sink.size());
// A character stream has to be told which charset produced the bytes.
dump("utf-8 ", decode(utf8, StandardCharsets.UTF_8));
dump("iso-8859-1", decode(utf8, StandardCharsets.ISO_8859_1));
}
static String decode(byte[] bytes, Charset cs) throws IOException {
StringBuilder text = new StringBuilder();
try (Reader in = new InputStreamReader(new ByteArrayInputStream(bytes), cs)) {
int c;
while ((c = in.read()) != -1) {
text.append((char) c);
}
}
return text.toString();
}
static void dump(String label, String s) {
StringBuilder hex = new StringBuilder();
for (int i = 0; i < s.length(); i++) {
hex.append(String.format("U+%04X ", (int) s.charAt(i)));
}
System.out.println(label + " -> " + s.length() + " chars: " + hex.toString().trim());
}
}Bytes are what storage actually holds; chars exist only after a charset decodes them, so the stream family and the charset together decide what your data means.
Worked examples
Binary data through a text path
Shows that copying bytes is lossless while decoding and re-encoding the same bytes as text destroys them.
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
public class BinaryThroughText {
public static void main(String[] args) throws IOException {
byte[] pngHeader = { (byte) 0x89, 'P', 'N', 'G', 0x0D, 0x0A, 0x1A, 0x0A };
ByteArrayOutputStream out = new ByteArrayOutputStream();
try (InputStream in = new ByteArrayInputStream(pngHeader)) {
in.transferTo(out);
}
System.out.println("byte copy: " + hex(out.toByteArray()));
String asText = new String(pngHeader, StandardCharsets.UTF_8);
byte[] roundTripped = asText.getBytes(StandardCharsets.UTF_8);
System.out.println("text round trip:" + hex(roundTripped));
System.out.println("size 8 became " + roundTripped.length);
}
static String hex(byte[] bytes) {
StringBuilder sb = new StringBuilder();
for (byte b : bytes) {
sb.append(String.format(" %02X", b));
}
return sb.toString();
}
}Example explained
Line 1in.transferTo(out) copies raw bytes, so 0x89 arrives as 0x89 and the byte path interprets nothing.
Line 2new String(pngHeader, UTF_8) meets 0x89, a UTF-8 continuation byte with no lead byte, and substitutes U+FFFD.
Line 3Re-encoding U+FFFD as UTF-8 emits EF BF BD, so 8 bytes become 10 and the original 0x89 is unrecoverable.
Line 4Nothing throws: the String charset constructor replaces malformed input instead of reporting it.
One character, two chars, four bytes
Shows that a single non-BMP character occupies two chars in a String and one multi-byte sequence on the wire.
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.nio.charset.StandardCharsets;
public class BridgeAndSurrogates {
public static void main(String[] args) throws IOException {
String clef = "\uD834\uDD1E"; // U+1D11E MUSICAL SYMBOL G CLEF
System.out.println("chars: " + clef.length());
System.out.println("codePoints: " + clef.codePointCount(0, clef.length()));
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try (Writer w = new OutputStreamWriter(bytes, StandardCharsets.UTF_8)) {
w.write(clef);
}
System.out.println("utf-8 bytes: " + bytes.size());
System.out.println("utf-16 bytes: " + clef.getBytes(StandardCharsets.UTF_16BE).length);
System.out.println("first byte: " + String.format("%02X", bytes.toByteArray()[0]));
}
}Example explained
Line 1clef.length() is 2 because U+1D11E is above U+FFFF, so String stores it as a surrogate pair of code units.
Line 2codePointCount collapses that pair back into the 1 character a human would count.
Line 3OutputStreamWriter encodes both code units as one 4-byte UTF-8 sequence beginning F0, not as two separate characters.
Line 4bytes.size() is only trustworthy after the writer is closed, since the encoder holds output in its own buffer.
What read() returns in each family
Compares the return values of InputStream.read() and Reader.read() over the exact same two bytes.
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.nio.charset.StandardCharsets;
public class ReadReturnValues {
public static void main(String[] args) throws IOException {
byte[] data = "\u00e9".getBytes(StandardCharsets.UTF_8); // C3 A9
try (InputStream in = new ByteArrayInputStream(data)) {
System.out.println("InputStream.read(): " + in.read() + ", " + in.read() + ", " + in.read());
}
try (Reader r = new InputStreamReader(new ByteArrayInputStream(data), StandardCharsets.UTF_8)) {
System.out.println("Reader.read(): " + r.read() + ", " + r.read());
}
}
}Example explained
Line 1InputStream.read() returns 195 then 169: the two UTF-8 bytes of the character, handed over independently.
Line 2Reader.read() returns 233 once, because the decoder consumed both bytes to build the single char U+00E9.
Line 3Both methods return int rather than byte or char so -1 can mean end of input without colliding with a real value.
Line 4Argument evaluation is left to right, so the three chained in.read() calls really do report the stream in order.
Important notes
Since Java 18 the default charset is UTF-8, but console output is a separate setting, so text can be correct on disk and still look wrong in a terminal; inspect the bytes with od -c or xxd before blaming the code.
InputStreamReader buffers ahead, so skipping a binary header with in.read() and then wrapping the stream is fine, but alternating between the two afterwards silently drops bytes.
Common mistakes
Calling new String(bytes) or FileReader with no charset argument: it works on the machine it was written on and then produces mojibake elsewhere, and before Java 18 the default charset varied by platform and locale.
Copying an image or archive with a Reader/Writer or via a String: invalid byte sequences become U+FFFD, the file size changes, and no exception warns you, so the corruption is discovered only when the file fails to open.
Assuming text.length() is the number of bytes to write, or sizing a byte[] from it: non-ASCII characters need 2 to 4 UTF-8 bytes, so the data gets truncated or cut in the middle of a character.
Try it yourself
Change, predict, then run
Build a byte array from the UTF-8 bytes of "na\u00efve", then decode that same array twice, once as UTF-8 and once as ISO-8859-1, printing each result's length() and each char as U+%04X. Write down both lengths before you run it.
Open the Java workspaceCheck your understanding
A program reads a UTF-8 file with new InputStreamReader(in, StandardCharsets.ISO_8859_1) and writes every char it reads back out through new OutputStreamWriter(out, StandardCharsets.ISO_8859_1). What happens to a file containing the word cafe with an accented final e?
- The program throws MalformedInputException when it reaches the 0xC3 byte.
- The output is corrupted: the accented character has become U+FFFD and cannot be recovered.
- The output file is byte-for-byte identical to the input, even though the in-memory string is wrong.
- The output loses one byte, because the two bytes of the accented character became a single char.
Show answer
ISO-8859-1 maps bytes 0x00-0xFF onto code points U+0000-U+00FF with no gaps and no illegal sequences, so decoding and re-encoding with it is a lossless round trip even when it is the wrong charset for the data. Option 1 is tempting because a mismatched charset usually does destroy data, but U+FFFD appears only when a decoder meets a byte sequence it cannot map, which cannot happen in Latin-1; the damage here stays in memory, where the string is 5 chars reading cafA-tilde-copyright and will compare, uppercase and measure incorrectly.