Hey all,

 

What would be the best approach to getting all unique values from a single column in a CSV file?

 

If someone provides a good answer I will include your name in my plugin information with a link to your site. An effeciant approach to my problem may help make this plugin far better so I'd be happy to give a little rec.

 

My wordpress plugin will deal with CSV files of over 500,000 rows of data and I need a list of unique values from a single column selected by the user. I guessed looping through all rows and all columns in the standard way then recording any new values found, will take far too long, or will it?!

 

Any ideas?

 

In the event you interested, my plugin will allow the uploading of CSV files which are then processed to create new posts with each row of data. The csv data does not have to be wordpress related and the main aim of the plugin is to allow the injection of affiliate data to blogs. The plugin is almost finished, allows users to assign CSV columns to parts of posts and now I need to allow assignment of category to specific data values i.e. if it is a book blog the CSV file data column for GENRE may be selected and unique values would be used to determine what category it goes in exactly.

 

Thanks everyone

 

Ryan

In PHP explode the CVS into an array and then use a function like array_unique to remove duplicates. That should give you a start

Right I get you. I'll consider that. I realised either way I do this, some users will still get it wrong but getting this problem sorted will minimise problems.

 

The plugin is nearly ready to use and will need a lot of improvements so I think this solution will be in a later update. Will try to remember to reply here on the outcome.

 

I have a csv file with 700,000 rows and my users will also so I might find a manual approach is the only approach but if users type in values that are not a match to the csv column then it just wont work. Imagine 100,000 posts in the wrong cagegory for example.

 

I can see people getting upset lol