Java: How to Break a List Into Batches Using Streams

Got a long Java List that you'd like to break down into a group of smaller, more manageable chunks?

Well then your favorite search engine brought you to the right place. I'll show you how to do it here.

And I'll give you a concise solution using a Java Stream API.

If you're unfamiliar with a Java Stream, it's an object that represents a sequence of elements. But it also gives you plenty of powerful tools that you can use to manipulate those elements.

That's why it works so well for this algorithm.

The Code

If you're just the copy-and-paste type, here's the code:

    public static <T> Collection<List<T>> createRows(List<T> inputList, int columnsPerRow) {
        AtomicInteger counter = new AtomicInteger();
        
        return inputList
                .stream()
                .collect(Collectors.groupingBy(gr -> counter.getAndIncrement() / columnsPerRow))
                .values();
    }

That particular method takes in a whole list of objects and spits back the chunkified Collection that includes smaller sub-lists.

The whole point of the method, in my case, is to create rows of objects that will get displayed on the UI. My UI uses Thymeleaf and I find it's easier to create rows on the server side than with the really bizarre Thymeleaf notation.

So if I've got 12 elements and I want to display those elements on rows with three columns each, I'll call that method above. The first parameter will be my List object of 12 elements and the second parameter will be the number 3.

Note the Type Safety

The method above works with any class. But it uses type-safety to ensure consistency at compile time.

That <T> you see next to static identifies the type that's going to be used. If you leave that out you'll get a compilation error right away because the compiler doesn't know what the heck T refers to everywhere else.

By the way: in my example, I'm using a String class. But again, you can use anything you want.

And yes, it's a utility class so I made the method static. That way other objects that use this class don't need to instantiate anything. Makes for tighter code.

Going Atomic

So what's with that AtomicInteger at the beginning of the method? Why not just use Integer?

Well, it's because we're eventually going to use that counter in a lambda expression. Take a look at Collectors.groupingBy() in the code above.

But you can only use final (or effectively final) variables in a lambda expression. That's for the same reason you can only use them in anonymous inner classes.

Theoretically, the method that created the lambda could terminate and leave the lambda (or inner class) running. So any variables created by that method are also gone.

That's why the lambda can't rely on a variable from that parent method at all. It really needs a constant.

Well, a constant won't work here because the code needs to increment the counter. Otherwise, it's not much of a counter at all.

Enter AtomicInteger. It's a handy tool when you need to increment a counter from many different threads. But it also works well in this case.

So why does AtomicInteger work when Integer doesn't? Because it's treated as effectively final.

That means the compiler thinks the value isn't changing even though the counter inside the AtomicInteger object really does change.

Dream a Little Stream of Me

Next, take a look at the Java Stream that gets created in the code:

        return inputList
                .stream()
                .collect(Collectors.groupingBy(gr -> counter.getAndIncrement() / columnsPerRow))
                .values();

That happens thanks to the stream() method from the List object.

And now that the code has that Stream object, it can start doing the good stuff.

The good stuff begins with a collect() method. That creates a Map object that groups the elements in the Stream according to the instructions specified by the given Collector.

That Collector object is instantiated with Collectors.groupingBy(). The groupingBy() static method takes a Function as its only parameter.

The Function appears as a lambda expression that's actually fairly intuitive. It increments the counter on the AtomicInteger object and divides that value by the number of columns per row (or number of elements per chunk, if that's how you roll).

Now here's the important part: it evaluates that function for each and every element in the sequence.

And what happens is the value returned by that function is the key in the Map object. And that key is expressed as an integer, the result of dividing the AtomicInteger value by the number of columns in the row.

The way Java works with this kind of calculation is that it disregards the remainder and returns the quotient by itself. So 0/3 will return 0 and 1/3 will return 0 and 2/3 will return 0.

But 3/3 will return 1. So will 4/3.

So if my columnsPerRow value is equal to 3, the first three elements in the Stream will get put in the Map object with a key of 0. The next three elements with a key of 1, and so on.

In doing that, the code groups the objects by threes into the Map object. 

But you don't want a Map object. You want a Collection.

And that's why you see .values() at the very end of that code block above. That will return just the values from the Map so you don't have to mess around with the keys.

You can test the whole thing out with something like this:

    public static void main(String[] args) {
        final List<String> list=new ArrayList<>();
        list.add("A");
        list.add("B");
        list.add("C");
        list.add("D");
        list.add("E");
        
        Collection<List<String>> rows = createRows(list, 3);
        System.err.println(rows);
    }

Run that and it will print this out:

[[A, B, C], [D, E]]

Try experimenting with larger lists.

Wrapping It Up

Is there a more efficient way of doing this? Oh, I'm sure some MIT wiz-bang could do it in a single line of code.

But this works just fine.

So now it's up to you to take what you've learned here and start chunking up your lists.

Have fun!

Photo by Sydney Troxell from Pexels