For a file of that size, you’ll need to use the background import, so my comments will relate to using that mode only.
In that mode, the limitations imposed by your php configuration are not memory, but file size limitations. You’re probably not running into that because it would fail to upload the file altogether. So you can rule that out.
The background import relies on the options table in the database to hold the queue of records to be processed. The kind of problem that you’re seeing is most likely due to caching of that table. This kind of caching is done at the server level…sometimes it will be under the control of a WP plugin, but often it is set up at the server configuration level. This table is often cached because it is very heavily used by WordPress.
What you need to do is suspend the caching, I can’t tell you how to do that, but you only need to do it temporarily for the purpose of importing the data. Make sure you’re working with the correct cache type: this will not be a page or browser cache, what you’re looking for is typically called an object cache, but it might also be a general HTTP (reverse proxy) cache, such as Varnish.
It is a good idea to use a small file for the purpose of testing the import, saves time and it’s easier to interpret the results. Also, it’s helpful to completely clear the database for each run so you can see exactly what is happening. This can be very efficiently done using phpMyAdmin, open the wp_participants_database table and perform a “truncate” on that table. It will leave all your configurations intact, while resetting the table to its initial state.