JAVA / CAPSTONE PROJECTS
Project: a concurrent file downloader with progress reporting
Build a thread-pool file downloader that counts bytes in one atomic counter and renders progress from a single reporter.
What you will learn
- Model each download as a Callable that owns its own buffer, stream and output file
- Funnel every chunk into one AtomicLong instead of a plain shared long field
- Render progress from a single reporter thread that samples, not from every worker
- Unwrap ExecutionException so one dead URL does not cancel the other downloads
Understanding Project: a concurrent file downloader with progress reporting
A downloader is a fan-out of independent blocking I/O jobs, so model one download as one Callable that owns everything mutable it touches: its own request, its own byte array buffer, its own destination file. The pool, not the task, decides how many run at once, and because these threads spend their time waiting on sockets rather than on CPU, a small fixed pool sized to what the server tolerates, or one virtual thread per download on Java 21, is the right shape. Share a single HttpClient across all tasks: it is thread-safe and pools connections, and building one per download throws away every keep-alive connection.
Progress exists only because you copy the body yourself. BodyHandlers.ofFile writes the file for you and tells you nothing until it has finished, whereas BodyHandlers.ofInputStream hands you the body stream and every read returns the exact count of bytes that just arrived; that int is your progress event. The denominator comes from the content-length header, which is legitimately missing on chunked or compressed responses, so the design has to survive an unknown total by showing bytes and rate rather than a percentage.
The reporting itself splits into a shared-state problem and a presentation problem. Every increment must land in one atomic counter, because total += n from four threads silently loses updates and a finished download then reports 93 percent; and one thread must own the rendering, because a carriage-return progress line written by four workers is unreadable, and printing on every 8 KB chunk of a 1 GB file means roughly 130,000 console writes competing with the download itself. Workers measure, a single reporter samples the counter on a timer and decides what the user sees.
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.atomic.AtomicLong;
public class Downloader {
record Job(String name, int size) {}
record Result(String name, long bytes, int chunks) {}
static final int CHUNK = 8192;
static final AtomicLong downloaded = new AtomicLong();
// Stands in for the real body stream:
// client.send(request, HttpResponse.BodyHandlers.ofInputStream()).body()
static InputStream open(Job job) {
return new ByteArrayInputStream(new byte[job.size()]);
}
static Result fetch(Job job) throws IOException {
byte[] buffer = new byte[CHUNK]; // one buffer per task, never shared
long bytes = 0;
int chunks = 0;
try (InputStream in = open(job)) {
int n;
while ((n = in.read(buffer)) != -1) {
bytes += n; // thread-confined, no lock needed
chunks++;
downloaded.addAndGet(n); // shared across threads, so atomic
}
}
return new Result(job.name(), bytes, chunks);
}
public static void main(String[] args) throws Exception {
List<Job> jobs = List.of(
new Job("alpha.bin", 5 * CHUNK),
new Job("beta.bin", 2 * CHUNK),
new Job("gamma.bin", 6 * CHUNK),
new Job("delta.bin", 3 * CHUNK));
long expected = jobs.stream().mapToLong(Job::size).sum();
ExecutorService pool = Executors.newFixedThreadPool(4);
List<Future<Result>> futures = new ArrayList<>();
for (Job job : jobs) {
futures.add(pool.submit(() -> fetch(job)));
}
pool.shutdown();
for (Future<Result> f : futures) {
Result r = f.get();
System.out.printf("%-10s %6d bytes %d chunks%n", r.name(), r.bytes(), r.chunks());
}
System.out.printf("total %d of %d bytes (%d%%)%n",
downloaded.get(), expected, downloaded.get() * 100 / expected);
}
}Parallel downloads must funnel their byte counts into one shared atomic number that exactly one thread turns into visible progress.
Worked examples
One reporter, many workers
Workers publish byte counts to a queue and a single reporter thread turns them into progress lines.
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class QueuedProgress {
static final int CHUNK = 8192;
static final long TOTAL = 15L * CHUNK;
static final BlockingQueue<Integer> events = new ArrayBlockingQueue<>(64);
static long fetch(int chunks) throws Exception {
long mine = 0;
byte[] buffer = new byte[CHUNK];
try (InputStream in = new ByteArrayInputStream(new byte[chunks * CHUNK])) {
int n;
while ((n = in.read(buffer)) != -1) {
mine += n;
events.put(n);
}
}
return mine;
}
public static void main(String[] args) throws Exception {
Thread reporter = new Thread(() -> {
long done = 0;
int seen = 0;
try {
while (done < TOTAL) {
done += events.take();
if (++seen % 5 == 0) {
System.out.println(done * 100 / TOTAL + "% " + done + "/" + TOTAL);
}
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
reporter.start();
List<Callable<Long>> tasks = new ArrayList<>();
for (int chunks : List.of(4, 6, 5)) {
tasks.add(() -> fetch(chunks));
}
ExecutorService pool = Executors.newFixedThreadPool(3);
pool.invokeAll(tasks);
pool.shutdown();
reporter.join();
System.out.println("all files complete");
}
}Example explained
Line 1events.put(n) is the whole contract between a worker and the display: workers report bytes and never format anything.
Line 2done and seen are plain local variables because only the reporter thread touches them, so no atomics or locks are needed there.
Line 3Every chunk here is exactly 8192 bytes, so the reporter crosses its 5th, 10th and 15th event at fixed byte totals and the output is stable no matter which worker wins the race.
Line 4invokeAll blocks until all three Callables finish and reporter.join() waits for the queue to drain, which is why the final line always prints last.
A failed URL among healthy ones
One download throws, the other two finish, and the summary still adds up.
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class PartialFailure {
record Job(String name, int size, boolean broken) {}
static long fetch(Job job) throws IOException {
if (job.broken()) {
throw new IOException("HTTP 404 for " + job.name());
}
long bytes = 0;
byte[] buffer = new byte[512];
try (InputStream in = new ByteArrayInputStream(new byte[job.size()])) {
int n;
while ((n = in.read(buffer)) != -1) {
bytes += n;
}
}
return bytes;
}
public static void main(String[] args) throws InterruptedException {
List<Job> jobs = List.of(
new Job("a.zip", 1200, false),
new Job("b.zip", 0, true),
new Job("c.zip", 900, false));
List<Callable<Long>> tasks = new ArrayList<>();
for (Job job : jobs) {
tasks.add(() -> fetch(job));
}
ExecutorService pool = Executors.newFixedThreadPool(3);
List<Future<Long>> futures = pool.invokeAll(tasks);
pool.shutdown();
long total = 0;
int failed = 0;
for (int i = 0; i < jobs.size(); i++) {
try {
long bytes = futures.get(i).get();
total += bytes;
System.out.println(jobs.get(i).name() + " ok " + bytes + " bytes");
} catch (ExecutionException e) {
failed++;
System.out.println(jobs.get(i).name() + " failed: " + e.getCause().getMessage());
}
}
System.out.println(total + " bytes downloaded, " + failed + " failed");
}
}Example explained
Line 1The IOException thrown inside fetch does not reach the pool's threads as a crash; the executor stores it in that task's Future while the other two downloads keep running.
Line 2futures.get(i).get() rethrows it wrapped in ExecutionException, so e.getCause() is the original IOException with the original message.
Line 3invokeAll returns futures in task order, so the report lines line up with the job list even though the downloads finished in some other order.
Line 4total counts only the bytes that really arrived, which is what lets the last line distinguish a partial run from a complete one.
Important notes
The code here reads from ByteArrayInputStream so it runs with no network. In the real project only the origin of the InputStream changes; the copy loop, the atomic counter and the reporter stay exactly as written.
content-length is absent on chunked and gzip-encoded responses, so firstValueAsLong returns an empty OptionalLong. Defaulting it to 0 makes the first percentage calculation throw ArithmeticException on integer division by zero.
Common mistakes
Hoisting the byte array buffer into a static field so all tasks share it: progress still climbs to 100 percent, but the threads overwrite each other's data and the saved files contain fragments of the wrong downloads.
Accumulating progress in a plain long field with total += n. The read-modify-write is not atomic, so increments are lost, a completed download reports something like 91 percent, and any loop waiting for total == expected never ends.
Computing the percentage in int arithmetic: once the byte count passes about 21 MB, bytes * 100 overflows and the display shows negative or nonsense percentages.
Try it yourself
Change, predict, then run
Add a fourth job of 5 chunks to the queue example and update TOTAL to match, then change the reporter to print on every 4th event. It should produce five lines, from 20% up to 100% 163840/163840.
Open the Java workspaceCheck your understanding
Four workers each call progress.addAndGet(n) after every read and each prints the returned value as a percentage. The download finishes correctly, but the console shows percentages that jump backwards, such as 61%, 48%, 73%. What explains it?
- AtomicLong.addAndGet is not atomic for long values on 32-bit JVMs, so some additions are lost
- Each thread caches its own copy of the counter field, so a thread only ever sees the bytes it downloaded itself
- A thread can be descheduled between updating the counter and printing, so a stale line reaches the console after a newer one; atomics order the updates, not the printing
- Integer division truncates, so percentages derived from a long counter cannot be trusted
Show answer
Each addAndGet returns a distinct, up-to-date total, but nothing ties a thread's println to the instant it read that total, so four threads printing independently can emit an older percentage after a newer one. The 32-bit tearing answer is tempting because non-atomic long fields really can be read half-updated, but AtomicLong is immune to that, and lost updates would leave the final total too low rather than make the display go backwards. The fix is not a bigger lock around the counter, it is moving rendering into one thread.