“Dataset Builder” Start Stop

Hi @dounsa,

Is it possible to STOP the “Dataset Builder > Run Bulk Generate ” (let’s say for example after 10% processing), then continue again from where it stopped with a separate dataset?

No yet, but I can add a pause button!

The reason I am asking, is that I have a lot of posts on my blog and I don’t want the “Dataset Builder”?to consume a lot of server resources while building the dataset.

In fact, this process doesn’t normally use your server resources; it’s mainly the OpenAI servers doing the job. During the process, I estimate that your server is sleeping 95% of the job, or even more.

Also, when analyzing my posts, there are sometimes similar questions but different answers. Should we clean these before uploading to open AI?

Absolutely. You need to make sure the data is nice and clean. It will influence the model by a lot. Which is part of the fun ??

What about WooCommerce products? Can we build a dataset for that as well?

I will add this in the Pro Version; in fact, it’s a bit tricky to read the content of a product, as I would need to layout the prices nicely depending on all the options and the plugins used to achieve this (it’s highly depend on each site). I’ll only be humanly-able to support this in a Pro Version. But sure, it’s part of my plan! ?? That said, developers can hook on the current process and make sure it works for their websites in the Free Version.

Thank you for the explanations.

I have noticed the following for example
Running the Dataset Builder>Run Bulk Generator on Posts (i have 1500+posts)
The progress bar moves very slowly and it took a few hours to to reach 1%
I stopped the generator by reloading the page.
Updated the entries and deleted some useless entries. Then Update with Defaults to get the green checkmark
Then Uploaded to OpenAI

Now, when I go back to continue to Run Bulk Generate on Posts again, it starts all over from 1st posts again. It does not skip or continue from where you stopped the last time.

I thinks there should be a solution for that since each Bulk Generate costs tokens on Open Ai

Hi @dounsa,

The progress bar moves very slowly and it took a few hours to to reach 1%

Not surprised; that process takes a loooooong time. Indeed, I know I need to find a solution to this. One quick idea would be to be able to restart from a certain post number, so let’s say it stops after 5 posts, you’ll be able to play it from #5. That’s easy to implement (the other solutions are much more complicated).

One quick idea would be to be able to restart from a certain post number, so let’s say it stops after 5 posts, you’ll be able to play it from #5. That’s easy to implement?

Excellent. If you can implement that, it will be amazing.

Okay! ?? It will be available in a few hours in the 0.9.5 release! ^^