Fred's ImageMagick Scripts



    Licensing:

    Copyright © Fred Weinhaus

    My scripts are available free of charge for non-commercial (non-profit) use, ONLY.

    For use of my scripts in commercial (for-profit) environments or non-free applications, please contact me (Fred Weinhaus) for licensing arrangements. My email address is fmw at alink dot net.

    If you: 1) redistribute, 2) incorporate any of these scripts into other free applications or 3) reprogram them in another scripting language, then you must contact me for permission, especially if the result might be used in a commercial or for-profit environment.

    Usage, whether stated or not in the script, is restricted to the above licensing arrangements. It is also subject, in a subordinate manner, to the ImageMagick license, which can be found at: http://www.imagemagick.org/script/license.php

    Please read the Pointers For Use on my home page to properly install and customize my scripts.

PHASHES


Computes and stores one or more perceptual hashes for all provided images.

Download Script

last modified: May 02, 2018



USAGE: phashes [-m method] [-d directory] [-t types] [-f file] [-s save] [-j jsonfile] [infile1 ... infileN]
USAGE: phashes [-h|-help]

-m ... methods ..... hash methods; one or more of: bmh, bdh, avh, pfh as a list of
.................... space or comma separate values or "all"; default=bmh
-d ... directory ... optional path to directory holding images; default is no
.................... directory; infiles have precedence over directory
-t ... types ....... types of file formats to process from directory as a space or
.................... comma separate list; default will process all files in the
.................... directory
-f ... file ........ optional text file containing a list of paths to input images
.................... with one image per row; directory has precedence over file
-s ... save ........ save the hash values to either the "comment" or "label" field in
.................... the image meta data or "neither"; default=comment
-j ... jsonfile .... json file to hold list of images, methods and hash values in
.................... addition to the save option specified
-p ... progress .... show progress to the terminal; choices are: 0 (nothing),
.................... 1 (image only), 2 (image and method) or 3 (image, method and hash);
.................... default=0; output can be redirected to a tab delimited text file
.................... that can be imported into Excel

PURPOSE: To compute and store one or more perceptual hashes for all provided images.

DESCRIPTION: PHASHES will compute and store one or more perceptual hashes for all provided images. If multiple images are provided, they can be listed in the commmand line or obtained from a directory or they can be provided via a text file containing a list one image per row using the -f option. There are four different hashes that can be used: bmh (block mean hash), bdh (block difference hash), avh (annular variance hash) and pfh (polar fft hash).

In all listed hashes, the images are first preprecessed to rec709luma grayscale removing the alpha channel and any virtual canvas removed. Then they scaled to a size of 256x256, stretched to full dynamic range and blurred with sigma=1.

Block Mean Hash (bmh): The preprocess image is scale to 16x16 pixels, which is the same as averaging non-overlapping blocks of 16x16 pixels. The hash is computed by comparing the global mean of the 256x256 image to each of the 16x16 pixel value. If the value is larger than the global mean, the hash is set to 1; otherwise 0. This forms a 256 length binary string for the hash.

Block Difference Hash (bdh): The preprocessed image is scale to 9x8 pixels. Then each pixel is subtracted from its neighbor to the right, thresholded at 0 and divided by 255. This forms a 128 length binary string. The preprocessed image is scale again, but this time to 8x9 pixels. Then each pixel is subtracted from its neighbor below, thresholded at 0 and divided by 255. This forms another 128 length binary string. Finally the two 128 length strings are appended to form a 256 length string for the binary hash.

Annular Variance Hash (avh): The preprocessed image is converted to 256x256 polar format with the columns representing angles and the rows representing radius. Thus each row is an annulus (circular ring) of the preprocessed image. The variance of each row is computed to get the variance as a function of radius as a list of 256 values. Each value is compared to the next one. If the next one is larger, then a value of 1 is assigned; othewise a value of 0 is assigned. This produces a 255 length binary string for the hash.

Polar FFT Hash (pfh): The preprocessed image is converted to 256x256 polar format with the columns representing angles and the rows representing radius. The FFT magnitude is then computed and the center 15x15 pixels are extracted and listed as 225 graylevel values in row first order. Each pixel's graylevel is then compared to the next one. If the next one is larger, then a value of 1 is assigned; othewise a value of 0 is assigned. This produces a 224 length binary string for the hash.

Attack Tests; Brightness, Contrast, Gamma, Blur, Compression, Color Noise, Grayscale Noise, Resizing, Rotation (cropped to original size), Flip-Flop-Transpose-Transverse, Watermark, Shear, Arc, Barrel, Pincushion and Other Images.

Pros and Cons:

Block Mean Hash is insensitive to all but geometry (rotation and distortion) attacks. Note that it is insensitive only to small rotations (<=5 deg) and small distortions, but none of the flip-flop-transpose-transverse attacks.

Block Difference Hash is insensitive to all but geometry (rotation and distortion) attacks. Note that it is insensitive only to small rotations (<=5 deg) and distortions, but none of the flip-flop-transpose-transverse attacks. It is slightly better than the Block Mean Hash with respect to the distortion attacks from my tests.

Annular Variance Hash is insenstive to all but the distortion attacks. It is especially insensitive to all rotations and the flip-flop-transpose-transverse attacks. It is insensitive only to very small amounts of watermark, shear and arc. It is sensitive to all barrel and pincushion attacks.

Polar FFT HASH is insensitive to all rotations, but not the flip-flop-transpose-transverse attacks. It is insensitive to all the other non-distortion attacks. It is insensitive only to very small amounts of watermark, shear, arc and barrel. It is sensitive to all pincushion attacks.

Test Results
http://www.fmwconcepts.com/imagemagick/phashes/phashing_mean1.pdf
http://www.fmwconcepts.com/imagemagick/phashes/phashing_diff1.pdf
http://www.fmwconcepts.com/imagemagick/phashes/phashing_annularvar1.pdf
http://www.fmwconcepts.com/imagemagick/phashes/phashing_polarfft1.pdf

References:
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=11&cad=rja&uact=8&ved=0ahUKEwjsvISLz5zaAhWrg1QKHQSJBJ04ChAWCCYwAA&url=https%3A%2F%2Fdigi.lib.ttu.ee%2Fi%2Ffile.php%3FDLID%3D2816%26t%3D1&usg=AOvVaw1qDKiIDQI80WZGUe7DbJEm
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=38&cad=rja&uact=8&ved=0ahUKEwjZzIXRt-rXAhVHrVQKHUOEDWc4HhAWCFUwBw&url=http%3A%2F%2Fjournals.dbuniversity.ac.in%2Fojs%2Findex.php%2FAJET%2Farticle%2Fdownload%2F15%2Fpdf_2&usg=AOvVaw03c7wUi1rCmj-uG25ea4fQ
https://pdfs.semanticscholar.org/b59b/14d3a0d7047a63cbbcfc25582fb915f60664.pdf
http://www.diva-portal.se/smash/get/diva2:946365/FULLTEXT01.pdf
http://www.phash.org/docs/design.html
https://tech.okcupid.com/evaluating-perceptual-image-hashes-okcupid/
http://qtandopencv.blogspot.my/2016/06/speed-up-image-hashing-of-opencvimghash.html
https://hal.archives-ouvertes.fr/hal-01222780/document
http://www.eurasip.org/Proceedings/Eusipco/2002/articles/paper745.pdf
http://radioeng-test.urel.feec.vutbr.cz/fulltexts/2007/07_04_076_081.pdf
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5118381/
https://www.cs.tau.ac.il/~amir1/PS/30_PolarFFT_ACHA.pdf
https://pdfs.semanticscholar.org/4c32/f6cab7cb79b70af00e2e2912d875bd58a799.pdf

ARGUMENTS:

-m methods ... hash METHODS. One or more of: bmh (block mean hash), bdh (block difference hash), avh (annular variance hash) and pfh (polar fft hash) as a list of comma separate values or "all". The default=bmh.

-d directory ... optional path to DIRECTORY holding images. The default is no directory. The infiles have precedence over the directory.

-t types ... limit of types of file formats to process from directory as a comma separate list. The default will process all files in the directory. This may not be a good idea if other than image files reside in the directory.

-f file ... optional text FILE containing a list of paths to the input images with one image per row. The directory has precedence over the file.

-s save ... SAVE the hash values to either the "comment" or "label" field in the image meta data. Choices are: comment (c), label (l) or neither/none (n). The default=comment

-j jsonfile ... optional JSON FILE to hold list of images, methods and hash values. The jsonfile is a simple text file that can be used in addition to the save option.

-p progress ... show PROGRESS to the terminal. The choices are: 0 (nothing), 1 (image only), 2 (image and method) or 3 (image, method and hash). The default=0. The output can be redirected to a tab delimited text file one row at a time that can be imported into Excel.

CAVEAT: No guarantee that this script will work on all platforms, nor that trapping of inconsistent parameters is complete and foolproof. Use At Your Own Risk.



Example

Original Images

  

Command To Create And Store Perceptual Hashes

phashes -m all -s comment test/lena.png test/barn.jpg
or
phashes -m all -s comment -d test

Retrieve Lena Data Command

convert test/lena.png -format "%c\n" info:

bmh 1100111110111100110001111011110001000011111110000100001111101001010001111111100101001110011100110100100011100011010000011110011101000001010001110100000111000111010000011100111101000000110011110100000011001010010000001111111001000000111101000100000111110000

bdh 0110010001110110011101000111000101111001010010010101101101011011000110110011001100010011010100110101101101011010010111000111110000000011110000010101111001111011110010000000011001110001100011000110011010011011010010000001101010000111111101001000001110010000

avh 111111110000000000000111000011111111111110001111111111111111000010100000000000000000000000011111111110000000011111111111111111111111111100111100000000000000000000000000000000101110101111100000000000100000000000011111111111111111111111111111111111111111111

pfh 11111010111111101111111010111110101110101111111111010101011101011010101111111111111010110101101010101101101110011111111011111010010110110101011111101110101010101011011011111011101111111010111111111111101101111011111110111111

Retrieve Barn Data Command

convert test/barn.jpg -format "%c\n" info:

bmh 1111111111111111111111111111111111111111111111111111111111111111111111011111101101111100111101110000110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000010000000000000001111101000000000000000001110111100100000

bdh 1111000011010000110010001100101011011001110010010100110101101110101010101010101110011101100110100111000001111100010111001000010110111110011111110000110010100010000000000000000000000000000000000111000011110001111111111111111011000111111111111111111000000001

avh 111111111110111111111111111111111100000000111111111111111111111111111111111111111000000001111111111111110000111111111000000000000000000000011111000000000011111110000000011111111111111000000110111111111111111111000000111101000000000000011111111110011100000

pfh 11111101111111111111101111111111111101101111110101101101110110101010111101110111101011010110111010001011010111101011011111011010101101111101101101011011101101111111111111101111111111111101111111111111111111111111111111111111

Command To Create And Store Perceptual Hashes In JSON file As Well As In Image Comment Field

phashes -m all -s comment -j lena_barn.json test/lena.png test/barn.jpg
or
phashes -m all -s comment -j lena_barn.json -d test

JSON File Output

{
	"PerceptualHashes":[
		{"lena.png":[
			{"method":"bmh", "phash":"1100111110111100110001111011110001000011111110000100001111101001010001111111100101001110011100110100100011100011010000011110011101000001010001110100000111000111010000011100111101000000110011110100000011001010010000001111111001000000111101000100000111110000"},
			{"method":"bdh", "phash":"0110010001110110011101000111000101111001010010010101101101011011000110110011001100010011010100110101101101011010010111000111110000000011110000010101111001111011110010000000011001110001100011000110011010011011010010000001101010000111111101001000001110010000"},
			{"method":"avh", "phash":"111111110000000000000111000011111111111110001111111111111111000010100000000000000000000000011111111110000000011111111111111111111111111100111100000000000000000000000000000000101110101111100000000000100000000000011111111111111111111111111111111111111111111"},
			{"method":"pfh", "phash":"11111010111111101111111010111110101110101111111111010101011101011010101111111111111010110101101010101101101110011111111011111010010110110101011111101110101010101011011011111011101111111010111111111111101101111011111110111111"}
			]},
		{"barn.jpg":[
			{"method":"bmh", "phash":"1111111111111111111111111111111111111111111111111111111111111111111111011111101101111100111101110000110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000010000000000000001111101000000000000000001110111100100000"},
			{"method":"bdh", "phash":"1111000011010000110010001100101011011001110010010100110101101110101010101010101110011101100110100111000001111100010111001000010110111110011111110000110010100010000000000000000000000000000000000111000011110001111111111111111011000111111111111111111000000001"},
			{"method":"avh", "phash":"111111111110111111111111111111111100000000111111111111111111111111111111111111111000000001111111111111110000111111111000000000000000000000011111000000000011111110000000011111111111111000000110111111111111111111000000111101000000000000011111111110011100000"},
			{"method":"pfh", "phash":"11111101111111111111101111111111111101101111110101101101110110101010111101110111101011010110111010001011010111101011011111011010101101111101101101011011101101111111111111101111111111111101111111111111111111111111111111111111"}
			]}
	]
}