• Resolved bt75007

    (@bt75007)


    I am trying to give my group access to our database online (website).
    I have approximately 375 data tables with a combined 6,000,000 records.
    I need to be able to query these records and generate a report in spreadsheet like format.
    Would I be better off with one giant table or several smaller ones?
    Ideas on the best tables or data handling plugin for this task?

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Author TobiasBg

    (@tobiasbg)

    Hi,

    thanks for your post, and sorry for the trouble.

    For data of this size, I would actually not recommend a plugin. Instead, you should look into using an actual mySQL database table for this. Not only will this give you the desired speed and performance, but will also offer the best methods for querying the data. This will however mean that you’d need to implement most of this as a custom solution.

    Regards,
    Tobias

    I had a very similar question and I looked for a suitable plugin that would allow me to have a mysql database in the backend instead of embedding data in a post as Tablepress does. I didn’t succeed in finding anything suitable, but I like Tablepress a lot, so I decided to add an extension to “fool” Tablepress into thinking the data came from a Tablepress table. My code is really only prototype standard, but basically it works as follows:

    1. 1Create a dummy table where the first column contains an sql statement – SELECT col1, col2… FROM table1
    2. Header row has column ‘sql’ and Footer row also (if you want to use Footer and Columns Filter capability)
    3. New shortcode is like [table id=1 sql=2 sql-filter='col1=value OR col2=value']
    4. The ‘sql’ setting is the row in which the sql statement exists in the table (it always expects it in column 1). I was able to use the same dummy table for many different sql statements.
    5. New extension (based on row-filters extension), calls the sql statement using wpdb and places the result in the $table[‘data’] array. It also sets the $table[‘visibility’] values accordingly.
    6. </eol>

      This solution worked for me because I have some very large tables – up to 50K rows – but I only need to display a subset of the data on any given page. The alternative was to divide my data into a few dozen smaller tables, but this would be very difficult to maintain in the long term.

      @tobiasbg , I can share with you what I have done if you want to private message me.
      Cheers,
      Catherine

    Plugin Author TobiasBg

    (@tobiasbg)

    Hi Catherine,

    that sounds really interesting, and I’d really like to take a look at this!
    Could you maybe email me the files to the email address that you already replied to yesterday? ?? Thanks!

    Regards,
    Tobias

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘One Large Table or Several Smaller Ones?’ is closed to new replies.