Checking Invisible Watermark used in Stable Diffusion library

If you checkout stable diffusion library for generating images, you will see that it uses a library named invisible_watermark. Since it raised my curiosity, I grabbed this library to check it out. You can install this library from pip using the following command.

pip install invisible-watermark

The watermarks generated by this library are truly invisible. Unlike normal watermark which gets embedded as a real text on the image with additional transparency, these are not displayed at all. This library uses discrete wavelet transform and discrete cosine transform to create and embed the watermark. You can get more details about these libraries on the project page.

In this blog, I will try to add watermarks to some images and try to see if we can recover the watermark after we do some regular operations on the images like resizing, color change etc.

Images and target tests

These are the four images that we will run the tests on. The park is in daylight. Niagara Falls is a little on the darker side. The other two images are patterns that normally can be used as web background.

We will of course use OpenCV for all our operations. After we embed the watermark, we will perform the following operations to see if the watermark is still available for read.

1. No operation of course 🙂
2. Reduce image to half the size
3. Make the image gray scale (However, we will still have it as a 3 channel image)
4. We will equalize histograms for each channel
5. We will normalize the image between 0..255

Implementations for individual operations

Let’s now put each of the operations we are going to do here. They all are pretty basic OpenCV routines.

Resize Image

def resize_image(self, img, scale) -> any:
    return cv2.resize(img, (0, 0), fx=scale, fy=scale, interpolation = cv2.INTER_CUBIC)

def resize_image2(self, img, width, height) -> any:
    return cv2.resize(img, (width, height), interpolation = cv2.INTER_CUBIC)

I have two methods for resizing images. I am eventually using the first method to scale. The second method I just used to create the merged image on top :). I can put the code later for anyone interested in that short numpy image stacking code.

Grayscale Image

def gray_image(self, img) -> any:
	grays = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
	# Returns a 3 channel image finally
	return cv2.merge([grays, grays, grays])

Histogram Equalize for each channel

def equalize_image(self, img) -> any:
	# Get channels
	b, g, r = cv2.split(img)
	# Equalize individual channels
	b = cv2.equalizeHist(b)
	g = cv2.equalizeHist(g)
	r = cv2.equalizeHist(r)
	return cv2.merge([b, g, r])

Normalize Image

def normalize_image(self, img) -> any:
	img_c = img
	cv2.normalize(img, img_c, 0, 255, cv2.NORM_MINMAX)
	return img_c

Watermark using the library

Now that all sub functions are out of the way, let’s discuss how we want to implement and validate the watermarking itself. Of course, these will use invisible watermarking.

The library itself supports three different modes of watermarking,

1. dwtDct: this will do DWT and DCT only. This is the fastest one and can be used real time.
2. dwtDctSvd: This will do DWT and DCT like the previous one. However, it will also do a SVD decomposition for each block.
3. rivaGan: Apparently uses a Hollywood model database to process. By default, I was not able to run it – so probably needs the model to be downloaded. At the end I just skipped this algorithm.

The following piece of code will do the watermarking,

def add_watermark(self, algo, txt, img) -> any:
	wme = WatermarkEncoder()
  	wme.set_watermark('bytes', txt.encode('utf-8'))
  	enc_img = wme.encode(img, algo)
  	return enc_img

I will send each algorithm to this block so that I can reuse the same code for each of them. To verify if the image watermark is still valid, we will use the following code,

def get_watermark(self, algo, bytesize, img) -> str:
  	wmd = WatermarkDecoder('bytes', bytesize * 8) # Total size of text has to be known
  	wm = wmd.decode(img, algo)
  	return wm.decode('utf-8')
  
def cmp_watermark(self, algo, orgtxt, img) -> None:
  	try:
      	dec_txt = self.get_watermark(algo, len(orgtxt), img)
      	if dec_txt != orgtxt:
        	print("\t\tERROR decoding: text mismatch: {}".format(dec_txt))
        else:
          	print("\t\tDecode Success: text match: {}".format(dec_txt))
	except Exception as e:
    	#print('\t\033[32;5mEXCEPTION received: {}\033[0m'.format(str(e)))
        print("\t\tERROR decoding: parse error")

So, we will use the comparison function on top to dump if we got a match. The first method gets the watermark from image.

Now let’s define some tuples so we can loop in one go,

imgbase = '../img/'
algos   = ('dwtDct', 'dwtDctSvd')
imgs    = ('niagara.jpg', 'pattern.jpg', 'blue.jpg', 'park.jpg')
ops     = [('Half Size', 'R'), 
           ('Gray Scale', 'G'),
           ('Equalized', 'E'),
           ('Normalized', 'N')]
fxtxt  = '@testing on: suturf.com'

Now we loop through each of these and start comparing and dumping results.

for imgp in imgs:
  	print("STARTING test with {}".format(imgp))
  	img = cv2.imread(imgbase + imgp)

  	for algo in algos:
    	print("\tEncrypting with {}".format(algo))
    	enc_img = app.add_watermark(algo, fxtxt, img)

    	print('\t--- Original Size:')
    	app.cmp_watermark(algo, fxtxt, enc_img)

    	for op in ops:
      		# Just missed 'match' by a version :)
      		if op[1] == 'R':
        		sim_img = app.resize_image(enc_img, 0.5)
        	elif op[1] == 'G':
          		sim_img = app.gray_image(enc_img)
          	elif op[1] == 'E':
            	sim_img = app.equalize_image(enc_img)
            elif op[1] == 'N':
              	sim_img = app.normalize_image(enc_img)
            else:
                sim_img = enc_img

            #cv2.imwrite(imgbase + algo + '_' + op[1] + '_' + imgp, sim_img)
            print('\t--- {}:'.format(op[0]))
            app.cmp_watermark(algo, fxtxt, sim_img)

Program Execution output

STARTING test with niagara.jpg
	Encrypting with dwtDct
	--- Original Size:
		Decode Success: text match: @testing on: suturf.com
	--- Half Size:
		ERROR decoding: text mismatch: 
	--- Gray Scale:
		ERROR decoding: text mismatch: 
	--- Equalized:
		Decode Success: text match: @testing on: suturf.com
	--- Normalized:
		Decode Success: text match: @testing on: suturf.com
	Encrypting with dwtDctSvd
	--- Original Size:
		Decode Success: text match: @testing on: suturf.com
	--- Half Size:
		ERROR decoding: text mismatch: 
	--- Gray Scale:
		ERROR decoding: text mismatch: 
	--- Equalized:
		ERROR decoding: parse error
	--- Normalized:
		Decode Success: text match: @testing on: suturf.com
STARTING test with pattern.jpg
	Encrypting with dwtDct
	--- Original Size:
		ERROR decoding: parse error
	--- Half Size:
		ERROR decoding: parse error
	--- Gray Scale:
		ERROR decoding: text mismatch: 
	--- Equalized:
		ERROR decoding: parse error
	--- Normalized:
		Decode Success: text match: @testing on: suturf.com
	Encrypting with dwtDctSvd
	--- Original Size:
		Decode Success: text match: @testing on: suturf.com
	--- Half Size:
		ERROR decoding: text mismatch: @
	--- Gray Scale:
		ERROR decoding: text mismatch: 
	--- Equalized:
		ERROR decoding: parse error
	--- Normalized:
		ERROR decoding: parse error
STARTING test with blue.jpg
	Encrypting with dwtDct
	--- Original Size:
		Decode Success: text match: @testing on: suturf.com
	--- Half Size:
		ERROR decoding: text mismatch: 
	--- Gray Scale:
		ERROR decoding: text mismatch: 
	--- Equalized:
		ERROR decoding: parse error
	--- Normalized:
		Decode Success: text match: @testing on: suturf.com
	Encrypting with dwtDctSvd
	--- Original Size:
		Decode Success: text match: @testing on: suturf.com
	--- Half Size:
		ERROR decoding: text mismatch: DD@@DDDD@DD@@DDD
	--- Gray Scale:
		ERROR decoding: text mismatch: 
	--- Equalized:
		ERROR decoding: parse error
	--- Normalized:
		Decode Success: text match: @testing on: suturf.com
STARTING test with park.jpg
	Encrypting with dwtDct
	--- Original Size:
		ERROR decoding: parse error
	--- Half Size:
		ERROR decoding: parse error
	--- Gray Scale:
		ERROR decoding: text mismatch: 
	--- Equalized:
		ERROR decoding: parse error
	--- Normalized:
		ERROR decoding: parse error
	Encrypting with dwtDctSvd
	--- Original Size:
		Decode Success: text match: @testing on: suturf.com
	--- Half Size:
		ERROR decoding: parse error
	--- Gray Scale:
		ERROR decoding: text mismatch: 
	--- Equalized:
		ERROR decoding: parse error
	--- Normalized:
		Decode Success: text match: @testing on: suturf.com

Result

Let’s put that output in a table.

In all case the image file generated were valid and can be successfully opened using image editors (Pixea in my case).

Conclusion

What I wanted to find out from the test is how good this tool is to maintain the identity of the original creator using the invisible watermark created. To conclude it looks like most of the regular operations on the image will make this watermark unavailable. Adding a physical watermark would still be a better option – however, in this case I do realize that physical watermark is not a feasible solution.

I think we have to keep looking till a better solution is found. Ciao for now!