Reading large CSV Files as a Collection in Laravel
Large-scale projects would often require to import data from large CSV files at some point. And if not handled properly, this can very well lead to memory issues.
Thankfully, you can use League’s CSV package to read CSV files conveniently and iterate over them as a collection in Laravel. Especially, using lazy collections. This way, you can read CSV files as a collection without loading the entire file into memory.
To get started, you can first install the league/csv package using Composer like so.
composer require league/csv:^9.0
Once done, you can use the League\Csv\Reader class to read the CSV file. You can pass the path to the CSV file to the constructor of the Reader class. This will return an instance of the Reader class.
use League\Csv\Reader;
$cities = Reader::createFromPath('path/to/cities.csv');
Now, you can set the header offset to 0 using the setHeaderOffset() method. This will set the first row of the CSV file as the header row.
$cities->setHeaderOffset(0);
Next, you can use the getRecords() method to iterate over the CSV file as a collection. This method returns a generator that can be used to iterate over the CSV file as a collection.
$cities->getRecords();
Next, you can pass in the generator to the LazyCollection::make() method to create a lazy collection. This will return an instance of the LazyCollection class.
And finally, you can use any of the methods available on the LazyCollection class to iterate over the CSV file as a collection.
For instance, you can use the groupBy() method to group the records by a specific column, the map() method to map over the records, the filter() method to filter the records, and so on.
Here’s putting it all together.
use Illuminate\Support\LazyCollection;
use League\Csv\Reader;
$cities = Reader::createFromPath('path/to/cities.csv');
$cities->setHeaderOffset(0);
$citiesCollection = LazyCollection::make(static fn () => yield from $cities->getRecords());
->groupBy('country')
->map(fn ($cities) => $cities->count())
->toArray();
As you can tell, we are using a static closure to create a generator that will be passed to the LazyCollection::make() method. This is because the LazyCollection::make() method expects a generator as an argument.
And that’s it. This way, you can read CSV files as a collection in Laravel without loading the entire file into memory.
I got to know about this courtesy of this tweet.
Like this article? Consider leaving a
Tip
👋 Hi there! I'm Amit. I write articles about all things web development. You can become a sponsor on my blog to help me continue my writing journey and get your brand in front of thousands of eyes.
