Duplicates in Search Results

  • Resolved gsdesign

    (@gsdesign)


    5 years, 6 months ago

    Greetings Jacob,

    I really appreciate your support and development of this plugin. I’m feeling like I’m bothering you, but someone within our company wants to know if there is a way to remove duplicates from searches. See, we have some of the same images with the same description and same filename in multiple galleries, and when you search, it picks up them all and puts them on the search results page. The person wants an image to only show up once in the search results.

    I noticed that someone asked about this 6 years ago, but there wasn’t a response. I don’t know if that was before you started working on the plugin, or if you have since developed a way to handle this. We are hoping there’s a way to do this.

    Even if this isn’t possible, I’d like to thank you for your work, its good.

Viewing 4 replies - 16 through 19 (of 19 total)
  • Thread Starter gsdesign

    (@gsdesign)

    Thanks for the suggestions.

    In answer to your question about the logfile, here is a copy and pasted sample (the user name has been starred out):

    Dbg: on:15.07.2019 11:13:10: ******: Items 318 and 313 are identical. Flags = 0x7000
    Dbg: on:15.07.2019 11:13:10: ******: Items 390 and 352 are identical. Flags = 0x7000
    Dbg: on:15.07.2019 11:13:10: ******: Items 496 and 495 are identical. Flags = 0x5100
    War: on:15.07.2019 11:13:10: ******: Items 669 and 557 have the same name, filename and description but score only 0 out of 6 possible matches. Flags = 0x040
    Dbg: on:15.07.2019 11:13:10: ******: Items 671 and 558 are identical. Flags = 0x7000
    War: on:15.07.2019 11:13:10: ******: Items 672 and 363 have the same name, filename and description but score only 2 out of 6 possible matches. Flags = 0x041100
    War: on:15.07.2019 11:13:10: ******: Items 674 and 366 have the same name, filename and description but score only 2 out of 6 possible matches. Flags = 0x041100
    Dbg: on:15.07.2019 11:13:10: ******: Items 677 and 362 are identical. Flags = 0x7000
    Dbg: on:15.07.2019 11:13:10: ******: Items 679 and 595 are identical. Flags = 0x3100
    Dbg: on:15.07.2019 11:13:10: ******: Items 679 and 597 are identical. Flags = 0x3100
    War: on:15.07.2019 11:13:10: ******: Items 690 and 495 have the same name, filename and description but score only 2 out of 6 possible matches. Flags = 0x041100
    War: on:15.07.2019 11:13:10: ******: Items 690 and 496 have the same name, filename and description but score only 2 out of 6 possible matches. Flags = 0x041100
    Dbg: on:15.07.2019 11:13:10: ******: Items 748 and 740 are identical. Flags = 0x7000
    Dbg: on:15.07.2019 11:13:10: ******: Items 747 and 742 are identical. Flags = 0x5100
    Dbg: on:15.07.2019 11:13:10: ******: Items 793 and 792 are identical. Flags = 0x5100
    War: on:15.07.2019 11:13:10: ******: Items 839 and 838 have the same name, filename and description but score only 0 out of 6 possible matches. Flags = 0x040
    Dbg: on:15.07.2019 11:13:09: ******: Items 931 and 862 are identical. Flags = 0x7000
    Dbg: on:15.07.2019 11:13:09: ******: Items 931 and 930 are identical. Flags = 0x7000

    In regards to what you mean by “Saving the Source Files” I’m not sure what you mean … can you clarify? At some point I manually reuploaded all the images via SFTP, both thumbnails and maybe what you mean as the source files, because I had to basically adjust every single file in the gallery (several hundred) and that seemed like the quickest solution to get them up. There’s the possibility that some of the “identitcal files”, since some were adjusted from the “source file” and some from a copy of the source file (it gets complicated because there were some errors and adjustments and distant duplicates along the way) don’t look exactly identical to your feature, so they show up as non duplicates except in title, filename and description. It’s a mess, but we’ve been using that gallery software now for over a couple of years, and there have been a lot of revisions to make everything work, along with changes to the materials.

    If you can loosen the parameters based on what I’ve written and what’s in the log files, that would be great. If not… well it’s understandable. It seems like an adjustment to the algorithm would be the most efficient way to solve the problem.

    Thanks so much for taking to the time to respond and assist with this feature and issue.

    Regards.

    Plugin Author Jacob N. Breetvelt

    (@opajaap)

    In regards to what you mean by “Saving the Source Files” I’m not sure what you mean … can you clarify?

    If Table IX-H1 and 2 are ticked, dut i think you do, because flag 0x4000 tells me that the source files are equal.

    I will change it this way:
    Minimum number of points required = 4.
    names equal and not empty: 1 point
    descriptions equal and not empty: 1 point
    filenames equal and not empty: 1 point (very old installations did not yet store orig filenames)
    EXIF unique ids available and equal: 4 points
    EXIF date/times available and equal: 4 points

    further checks on equality, each one point:
    – source file sizes
    – display file sizes
    – thumbnail file sizes
    – source file contents
    – display file contents
    – thumbnail file contents

    Thread Starter gsdesign

    (@gsdesign)

    Hey Jacob,

    Thanks again for the quick response and the work you are doing. It is saving the Source Files, both IX-H1 and IX-H2 are checked.

    Hopefully that new system will work. Looking forward to the next release!

    Regards.

    Plugin Author Jacob N. Breetvelt

    (@opajaap)

    Pls update to 7.2.03 and look again.

Viewing 4 replies - 16 through 19 (of 19 total)
  • The topic ‘Duplicates in Search Results’ is closed to new replies.