Secure Computing: Sec-C

I think that the US Postal Service is finally nearing the end of its death spiral. The USPS recently asked Congress to alter the existing laws: they only want to deliver mail five days a week (instead of six) and they want to increase stamp rates, again.

Online Bill Pay

The postal service has a couple of serious issues. First, they are not generating enough revenue to cover their operations. Postmaster General John Potter estimates that the USPS faces a cumulative loss of $238 billion over 10 years. I don't doubt the numbers -- it's probably in the ballpark.

The second issue is a competitive disadvantage. With nearly all banks and utilities offering online bill pay services (and usually for free), people have realized that they don't need to use stamps. Why should I pay $0.44 a month to pay a bill, when I can pay it for free! I have my cable bill, phone bill, long distance phone bill, cell phone bill, credit card bill, electricity, water, gas, and sewage bills. That's 9 bills per month at $0.44 cents per bill. Paying online saves me $47.52 per year in stamps that I no longer use! Four years ago, I used up a checkbook each year. Last year? I wrote a total of 8 checks. That's an additional savings of $15 per year since I no longer have to order replacement checks!

In addition to the dwindling number of bill payments sent through the post office, there is also the dwindling number of personal letters. Email, cell phones, SMS/texting, blogs, twitter, and other social media services have effectively made personal letters obsolete. The only time I really see personal letters anymore are when they are accompanied by birthday and holiday greeting cards.

How can the USPS compete against on-time bill payment services and personal communications that are near real-time and effectively free? They can't.

Spreading Out

Most companies learned long ago that vertical markets are limited. It is too easy for a competitor to cut off your customer base. With the postal service, they first faced competition from professional package delivery companies like UPS and FedEx. Then they met the Internet, which effectively made most USPS services obsolete.

The USPS has tried a couple of ways to enter other markets. They came up with home-stamp printers, so you don't need to buy stamps at the post office. I only know two small companies that bought these, and they stopped using them because the ink was too expensive. And while the idea of creating custom stamp pictures was cute, I haven't seem them used with the exception of one wedding invitation. Frankly, the price of $5-$10 over the cost of the stamps was just too expensive.

Not all of the USPS ideas have been bad ones. For example, they offer flat-rate packaging. Regardless of the item (up to 70 lbs), if you can fit it into their 12.5"x9.5" envelope then you will pay $4.90. They also have small, medium, and large flat-rate boxes. These are ideal for those eBay packages, or for shipping off computer supplies! The prices are very competitive compared to UPS and FedEx.

Thinking Small

Unfortunately, the USPS has been unable to think beyond "mail delivery". For example, ten years ago they proposed an offering to forward your postal mail to your email address. Uh, why? Why pay for a conversion service when I can just ask the sender to email it directly? And more importantly, who will be typing in the letter? I don't want anyone else to read my mail!

As with any industry, there are only three ways to increase revenue: create more offerings, increase prices, or decrease costs. The USPS has clearly failed to create more offerings. The services that they currently offer are, for the most, not competitive. (We don't call it "snail mail" for nothing!) Thus, they only have two other options.

Last year, the USPS suggested reducing mail delivery from six days a week to five. They just brought up the idea again. I can see the pros and cons to this. On one hand, most businesses shut down over the weekend. There is no real reason to deliver mail on Saturday to companies that are not open. And I don't think most residences will really care if no mail is delivered on Saturdays. As with banks, you can still do transactions over the weekend (mail letters), but the transaction will no complete until the weekday.

On the other hand, 5-day-delivery puts the USPS at a serious disadvantage. Both FedEx and UPS have Saturday delivery options. If it absolutely needs to be there, then the USPS goes from a weak option to no option.

Finally, there is the option to increase prices. The USPS began seriously increasing prices in the early 1970s, and the practice has since become a run-away process. Each of the last four years has seen a price increase.


Why are the early 1970s important? That's when the USPS went from a government-run organization to a semi-independent corporation. So the USPS went corporate and began increasing stamp prices. At the same time, they failed to address the growing threat from the Internet. Every time they increase stamp prices, they reduce the number of people sending letters and force more people to use online services that do not use the USPS.

Thinking Outside the (Flat-Rate) Box

It still isn't too late for the USPS to recover from this massive loss. But they need to think differently. For example:

• Free Services. Every FedEx and UPS package includes a tacking number. In contrast, the USPS charges extra for tracking. They need to provide free tracking in order to remain competitive. I want the option to mail an untracked letter, but I also want the option to track letters for free. That blurry stamp with an unreadable postmark is no longer a viable option. I want to go to usps.com and get a tracking number, write it on my envelope, and mail the letter! I want to see that it was picked up at 9:45am and delivered four days later.
  
  
• Simplify Services. If I want to send a letter, I need to write the address on the letter. If I want it insured, then I need to fill out a form. If I want it certified, then I need to fill out another form. Compare this with UPS and FedEx: one form and check what you want. Why is the line at the post office so long? Customers are constantly filling out additional forms! The USPS needs to simplify this process. Simplification will lead to faster service, happier customers, and less cost overhead.
  
  
• Usability! My local post office has four teller windows (of which, only 1 or 2 are usually staffed at any given time, leading to long lines) and two automated package mailing systems. The two automated systems are virtually unused. One automated system may have one customer at it, while the human tellers will have a line of 30 or more customers. This is a massive waste of a good opportunity.
  
  
• Additional services! The USPS needs to branch out! Offering a PO Box address is great! How about a service to email me when the mail has been delivered to my box? Maybe even an email about any packages that arrive or when the PO Box is overflowing? I want to be able to set an alert, so anything from California or Amazon immediately generates a notification to me! And why can't the USPS sign for my UPS or FedEx packages? I'd rather have the packages sit in the post office than in plain sight on my front porch.
  
  
• More locations! If I want to mail a package, then I need to go to the post office. (Assuming I cannot weigh it myself, compute the postage amount, and fit it into my tiny out-going mailbox.) The USPS only has five post offices in Fort Collins. In contrast, FedEx and UPS have lots of locations where I can ship packages. Besides the official FedEx and UPS buildings, I can go to MailBoxes Etc, Postal Plus, Kinkos, and even the grocery store! Most big companies have scheduled FedEx/UPS pickups, and I can even call them up and schedule a pickup. My time costs money. I'm willing to pay a little more for the convenience factor. (If I earn $60/hr, then a 30-minute wait in line at the post office costs $30. I can ship a huge package via FedEx and have them pick it up for less.)

In effect, the USPS needs to give me a reason to want to use their services. Without a new reason, they cannot compete against UPS, FedEx, and the Internet. If they don't change their ways, then snail mail will become obsolete.

The Down Side

So let's say that the USPS goes out of business. (From 6-day delivery to 5-day, to 3-day, to none.) What's the problem with that? Well, here's just a few issues:

• Anonymity. You cannot mail a letter anonymously through FedEx or UPS. Sure, you can put down a fake sender address, but that isn't the same as leaving it blank. Got a hot tip for the police? Want to complain about your boss? Need to mail a ransom letter? The USPS is your best option for anonymity.
  
  
• Slow Down. Sometimes you don't want next-day delivery. Mailing home stuff you collected at a conference or bought on a trip? You don't want it to arrive before you do. Want to float a check so it gets delivered the day after you get paid? The USPS is a great option if you do not want it to arrive immediately. While some auto-pay systems do allow you to schedule the delivery date, you usually cannot schedule a payment when you have insufficient funds. And FedEx and UPS don't have an option to "deliver it next month". (For next-month delivery, use USPS bulk/book rate.)
  
  
• Hold mail. Going away on a trip? I really like the "hold my mail" yellow forms. I only wish that there was a way to tell FedEx and UPS to hold delivery until next Tuesday.
  
  
• No Internet. The best advantage for the USPS is that I don't need Internet access. If they go away, then there is no good option for paying bills without an online bill-pay service. If, like my grandmother, you don't have Internet access, then you lose the ability to pay bills and contact relatives. Requiring online access places more power into the hands of the telcos. As long as there is a non-Internet option, the telcos must remain competitive. Remove the non-Internet option and Internet providers become a necessity -- watch them increase their rates.

While the USPS does have unique offerings, their management seems hyper-focused on the vertical delivery market. If they want to survive, then they need to offer more services and lower their costs -- not cut services and increase prices.

There are many different types of mass media manipulations. The simplest are outright lies. For example, John Edwards first claimed that he did not father his mistress's child. But nearly three years later, he admitted the truth.

But how do you cover up a big problem? For decades, governments buried documents related to UFOs. It was not until 2008 that the UK began releasing government reports, and they are still releasing documents. (I'm not choosing a side in the UFO debate -- I'm only showing that there was a cover up and not delving into what they were covering up.) Another common tactic is to claim ignorance. Did Bill Gates really not remember sending emails that demanded a link between advertising agreements with ties to their web browser? Did Toyoda really not know about the Toyota accelerator problem until recently?

The most intriguing examples of manipulation are the active cover ups. Did you hear about the train derailments that have been happening for decades? Of course not. There is no need to cause panic regarding domestic terrorism. (Most train derailments never make it past the local news, even when hazardous chemicals or munitions are involved, regardless of whether the cause was accidental or something else.) And remember the baseball steroid use controversy? Every single network news channel covered this boring congressional hearing rather than the heated debated on extending the Protect America Act.

However, the most amateurish cover-ups are the ones where they try to rewrite history after it has been made public. Iran does this all the time: Everybody loves Ahmadinejad and Iran did launch missiles! (Their pictures prove it!) And Neda killed herself; she was not shot by the government. At least, that's how they want to rewrite history.

And along with Iran, we now have SeaWorld.

Jonah? Are you in here?

First, the facts: On 24-Feb-2010, an orca whale named Tilikum killed a trainer, in front of a crowd of people. This same whale has been associated with two other deaths, including another trainer.

All of the witnesses seem to have the same story -- and they went live with their stories very quickly. They say that the whale grabbed the woman by her waist, shook her, and dragged her under the water.

"Queequeg! Ready yar harpoon!"

It did not take long for SeaWorld to render their own version of the story. First sheriff's spokesman Jim Solomons claimed that trainer Dawn Brancheau fell into the tank.

Then SeaWorld's curator of zoological operations, Chuck Tompkins, said that the whale grabbed her by her ponytail and pulled her underwater. I've known many women with long hair. It should be pretty easy for witnesses to distinguish "grabbed by the hair" from "grabbed her around the waist."

While this certainly appears to be a tragic accident, SeaWorld is doing a very poor cover-up. The story keeps changing and it does not match witness accounts.

Hard To Port!

When caught in a lie and an ever-changing story, there are really only two alternatives: admit to the wrongdoing, or divert attention. Perhaps this is why SeaWorld Parks & Entertainment President Jim Atchison said that this incident "has been vastly overplayed within the media." And General Manager Dan Brown refused to take questions at a press conference, saying "Please bear with us, we've just lost a member of our family."

While the loss is heartbreaking for her family and friends, SeaWorld is not a family -- it is a corporation. (Owned by Anheuser Busch, recently bought out by Inbev.) The right to privacy given to families does not apply to corporations. If a person is killed at a public event, even by accident, the company has a duty to quickly and accurately define the situation.

Frankly, I would not mind if SeaWorld simply said that they were still investigating or were waiting for the coroner's report. But that is not what they have done. They have attempted to alter history -- "she slipped into the pool" and "she was grabbed by the hair" are not the same as "she was grabbed by the waist" or by the arm (one witness said it could have been her arm, but none of the witnesses have said that it was her hair). This was a tragedy, but SeaWorld has attempted to rewrite history, likely to shift the blame from an unsafe animal with a history of killing people to the trainer.

Thar She Blows

The real question that I have: Most animal parks record every show and every moment that an animal is with a trainer. This is done in case of an accident. In the event that somethings goes wrong, they can review the footage and change their procedures in order to prevent it from happening again. If a tiger or an elephant attacks their trainer, there is footage that can be reviewed. Did the trainer do something wrong? Did they miss some sign of danger? Or was the incident an unprovoked attack from a wild animal?

Considering that alarms and sirens went off immediately, you know that staff members were watching. So where is the video? I'm not asking CNN to air Dawn's last moments. I am asking for clarity. What is SeaWorld covering up? (I'm guessing liability. If it was the handler's fault, then they probably don't have to pay the family. But if it was the animal, then there are some hard decisions to make.)

NOTE: This is not a discussion about keeping wild animals at amusement parks, or even about what should be done with the whale. Everyone has their own inflexible opinion. (Personally, I think the whale is too dangerous and should be let free... near Japan where they still do whaling. Kind of like Running Man on the high seas.) Instead, this is a discussion about corporate media manipulation.

I usually bring a book or a magazine with me when I travel. Since airplanes forbid the use of electronics during the beginning and end of the flight, and turbulence can kill hard drives, old fashioned printed paper is a good way to pass the time. I try not to bring something too deep (e.g., advanced calculus or particle physics) or too shallow (Newsweek or Time). Occasionally I'll buy a copy of 2600: The Hacker's Quarterly. However, I was so disappointed with the latest issue, I think I won't buy one again.

I don't buy 2600 regularly. Most of the time the articles are worthless. But if one or two articles are interesting, then I'll get a copy to bring on the airplane. However, the most recent issue (Winter 2009-2010) actually managed to offend me. I am offended when a hacker magazine advocates stupidity and activities that are unethical at best and potentially illegal.

Magazine Shopping

I should have known better. They say that you cannot judge a book by its cover. But in this case, the photoshopping is so horrendous as to be offensive. It really should have been a clue to me...



Just a few of the problems:

• The drop switchboard is floating. It's only visible leg casts no shadow. It also isn't plugged into anything.
  
• There are shadows on the floor between the woman and the switchboard, but they just fade away. Nothing is casting the shadows.
  
• The baseboard changes angle as it passes the woman. Yet the wall behind her shows no corner or bend.
  
• The woman is wearing a headset. The chord actually becomes thin and faint as it passes the name tag.
  
• Does the woman have an adam's apple?
  
• The edges of the "photo" have been drawn so as to appear old, with dust and water stains. Yet the artist forgot that photo damage virtually never stops right at the edge of the picture. The picture itself has no damage.

While 2600 (both the magazine and radio show) has never been known for high quality, this lack of attention to detail is even startling by their standards. (2600 usually photoshops their covers, but this is the first one that is really, amazingly bad.)

What about the contents?

Just reading the articles, I was amazed by the amount of bogus information. For example, the first article was "Pwning Whole Disk Encryption" by m0untainrebel. Basically, he describes a weakness for whole disk encryption. The weakness? If you have physical access to the hard drive, then you can place a boot sector virus and capture the decryption password.

Let's backup for a moment... If you have covert, physical access then you already pwn the system. You can install a keyboard logger, internal HD bus intercept, video camera to watch the victim enter the password, or even malware on a USB drive plugged into the back of the computer (where the user will never notice). Worst case? You can mirror the drive and crack the password at your convenient off-site lab.

And remember: this is a lay-and-wait strategy. It must be covert. If the victim suspects that you did anything to the computer, then they are unlikely to login. Law enforcement would probably not use this technique -- it would require a very hard-to-get court order and has a high risk of failure (since it requires stealth). Considering that it bypasses a security mechanism, I cannot envision any legal reason for non-law enforcement to use this technique. Thus, the author not only overlooks the obvious (physical access means access), he also appears to promote a technique that can only have unlawful purposes.

Frames...

Another article was even worse. In "Revenge is a Dish Best Served Cold", Valnour describes the way he got back at a high school bully. He was being harassed via Windows net send messages. So... he went to class early and planted a program on the bully's computer to send a stupid insult to the entire class (including the teacher). The result? The bully was suspended.

This article shows a clear lack of ethics and a lack of the hacker mentality. First off, you don't frame someone for something they didn't do. While the bully may have used net send, he was not responsible for sending the message to the class -- that was Valnour who did it. The bully was falsely accused of sending the message that got him in trouble. It does not matter that he sent other messages to Valnour; the bully was still falsely accused of a crime that he did not commit. Valnour has no sense of ethics.

Also, Valnour planted the code manually and after-hours. This is the mentality of script kiddie and not a hacker. Things a hacker would have considered:

• Valnour said that each computer had a sticker that identified the hostname and IP address. The obvious hack: swap stickers with the teacher.
  
  
• If you are there after-hours and have physical access, then modify the DNS server or swap IP addresses so a network resolution would yield the teacher's computer and not your computer. (You can also edit the hosts.conf file and/or add in a specific routing table entry.) Now when the bully sends his message to "you", it will actually go to the teacher. You might have reconfigured the computer, but he was the one who sent the message -- now he isn't framed for something he didn't do.
  
  
• net send usually uses RPC over UDP but can use TCP. Since you know you are the target, setup a redirector so any IP requests from the bully's computer receive an ICMP redirect packet that tells his computer to resend the message to the teacher's computer. Again, it isn't a frame since he is the one sending the message. You're just telling his computer to send it somewhere else...
  
  
• If you're going to login to his system, at least have the guts to do it while he's there! Don't be a coward, planting malware on his system during off hours. Even if you lack the ethics, a real hacker would login remotely, open a window on his desktop (so he can see it happening and feel panic) and send the message while he watches helplessly. Sure, it's a frame-up. But at least it isn't cowardly.

Sex as a weapon

In "Social Engineering from a New Perspective", Lilith found out that she can flirt her way to passwords and unauthorized access. Wow! Incredible! I bet no other woman has ever realized that this is possible! What Lilith fails to understand is that flirting is a very superficial method of social engineering. As Kevin Mitnick has repeatedly shown, social engineering is about confidence and not looks. If the only tool you have are your looks, then you won't get far with social engineering.

I should also point out that flirting is not limited to females. In a red-team attack, I actually used "chit chat" (as Lilith calls it) to keep a female defender occupied while other red-team members altered the contents of her open vi cache while she was editing the password file. (This was an awesome hack with social engineering as a delay tactic.) When she saved the file, the altered cache was written to the password file.

A little too late

Other articles were equally inane. For example, a person called "dolst" wrote about an Adobe side effect ("Hey Adobe! Leave My Boot Loader Alone!"). On Windows systems, some Adobe products store the product serial number in the boot sector. If you have a dual-boot system (e.g, Linux with GRUB in the boot sector), then this can corrupt the boot loader.

This would have been a really fascinating article... if it wasn't already known since at least 2004. (That's more than half a decade for you 2600 fans.)

Looking for Literature

With 2600 no longer a viable option, what do other people read on airplanes? I'm looking for something technical but not deep (short articles rather than a novel), interesting, and that doesn't require power during takeoffs and landings.

The last few weeks have been a serious rush. I think I can summarize it simply: newer isn't always better.

Spinning Down

A few months ago I lost "yet another" hard drive. Fortunately, it was part of a RAID, so I didn't lose any data. (A lesson I learned from my first hard drive failure -- always use a RAID.) I seem to be getting 2-3 years out of newer hard drives, and it does not matter which manufacturer created the drive.

I have a few old computers collecting dust in the back room. Recently I had a need for some software that I wrote back in the 1990's. I couldn't find a copy on my newer systems, but I knew it was on the old, dusty box. I plugged it in, powered it on... and it came up without a problem. Now, to put things into perspective: the hard drive is a 120 MB (yes, megabyte) Conner drive. I acquired it around 1990. This drive ran continuous duty for over 15 years before being powered down and archived for five years. And... it powered back up without a problem.

When it comes to hard drives, I plan for new ones to fail -- because they will fail. But old hard drives? I think my Conner could easily do another ten years continuous duty. (Too bad it is only 120 MB!)

Broken Windows

The newer X-Windows server (since about 2008) is much more automated. In Ubuntu's Karmic Koala (9.10), it does not even include an Xorg.conf file -- the entire configuration is automatically detected.

The good news is, the X-Server will likely configure itself correctly and start up without a problem. The bad news is, if it has problems, then many of the debugging tools that you will need are broken. Making matters worse, they have been broken for years.

A good example is the xvidtune program. If you have a flat screen monitor, or even a newer tube monitor, then it will likely auto adjust the frequency and center the image on the screen. But if you have an older monitor, then you may need to manually align the desktop's position on the display. Depending on the video card, monitor, and auto-detected X-Windows settings, the desktop may need more shifting than the monitor's manual controls allow. The real solutions is xvidtune, which allows you to adjust the position on the display by tweaking the horizontal and vertical frequencies.

Unfortunately, xvidtune has been broken for years -- since X-Server version 1.4 (2007). And while plenty of people have reported the problem, it has remained broken for at least three years.

HTML Doc

I've been doing a lot of technical documentation lately. I'm writing it in HTML and using htmldoc to convert it to PDF. The problem is, my older Ubuntu Dapper Drake system could generate the docs but all of my newer systems could not. It turns out, my HTML includes arrows for menus (–› created using –›). On the newer systems, they just print blank spaces.

I eventually traced the problem to the version of htmldoc. Version 1.8.24 works fine, but the newer versions (1.8.27 through 1.9) seem to have problems with ampersand codes.

Et Tu, JPEG?

For my image analysis stuff, I rely on the FreeImage library for loading most image formats and saving all formats. (FreeImage has a few quirks with corrupted files, so I wrote my own libraries for loading some file formats.)

I recently upgraded from FreeImage 3.11.0 to 3.13.1... and immediately noticed some problems. The Error Level Analysis and color space algorithms were giving different results for some of my regression tests. I even tried 3.12.0 and 3.13.0 -- and found the cutoff: 3.12.0 renders JPEGs correctly, 3.13.0 does not.

FreeImage actually uses the library provided by the Independent JPEG Group (IJG). FreeImage 3.12.0 uses jpeglib v.6b, while 3.13.0 upgraded to jpeglib v.8. Somewhere between 6b and 8, IJG did a significant rewrite to their library for applying chrominance. The net result: JPEGs rendered by IJG's jpeglib v.8 no longer look like JPEGs rendered with other libraries (IJG and non-IJG).

Don't get me wrong: The pictures still look like pictures, the differences are subtle, and the changes really only impact extreme corner-cases. However, if the library does not render colors in those corner cases exactly like other libraries, then I cannot use it. Good thing I could easily regress to 3.12.0.

Blast From The Past

Not everything old is better than their newer counterparts. My iPod is a much better MP3 player than my old no-name brand player. My USB LED mouse is far superior to the old serial mouse (if for no other reason than the wheels don't get gummed up). And my netbook is a huge improvement over my old Dell laptop.

But in the last few weeks I have been repeatedly reminded that newer is not always better. (And don't get me started on the Toyota recall. Good thing my car is old...)

In my previous blog entry, I discussed how JPEG is widely known as a lossy format and the two causes of the loss: coloring and quantization (Q) tables. The Q tables are what lead to continual data loss every time you resave an image. However, not everyone understands how the data loss from Q tables impacts the image.

But I Saw It On YouTube!

Chris Hanson recently pointed me to a YouTube video that claims to show what happens after a JPEG image is resaved 500 times.


The video starts with a picture of my next wife, actress Alyson Hannigan, and shows it seriously degrade over the course of 500 resaves.

There's a problem here: the visible artifacts. This isn't how JPEGs works. The video, which claims to have resaved the JPEG image 500 times, is doing something other than "JPEG".

Converting to Frequency

To understand the kind of data loss from JPEG Q tables, you need to understand how Q tables work.

The image is divided into 8x8 pixel squares. The 8x8 squares are converted into scalars for 64 frequencies. The 64 frequency basis functions look like these:


So let's say we have 64 frequency scalars like:

  -49 -145  112  -66   15  -39   20    0

   13   55    8  -30  -43   44   -9    6

 -161 -123  -58   85   13   10   -7    0

   30  115    3   20  -14   -6    9    0

   18   69  -26  -19   23   11   12    0

  -27   -2  -24   -6   10  -11    1    1

   15   35    7   -8    0    0    0    0

    0    7    0    0    0   12    0    0

(I didn't make these values up -- they come from the red channel 8x8 square at 216x152 to 223x159 in the image below -- her eye.)

So what this means: take the first basis frequency (solid white) and scale it by -49. Add to it the second basis frequency (white/black) multiplied by -145, and so on. The total sum of scaled basis functions yields the actual color.

Q Tables

The top-left basis function (solid white) represents the lowest frequency range. In contrast, the bottom right (checkerboard) is the highest frequency range. Since the human eye is not very sensitive to high frequencies, Q tables are used to reduce the values.

Let's say we have the Q table:

   12    8    8   12   17   21   24   17

    8    9    9   11   15   19   12   12

    8    9   10   12   19   12   12   12

   12   11   12   21   12   12   12   12

   17   15   19   12   12   12   12   12

   21   19   12   12   12   12   12   12

   24   12   12   12   12   12   12   12

   17   12   12   12   12   12   12   12

(Again, not made up. It comes from the image below.)

To apply it, divide each scalar by the associated Q value. For example, -49/12 = -4.08. Since JPEGs use integer math, this becomes -4. The total table becomes:

   -4  -18   14   -5    0   -1    0    0

    1    6    0   -2   -2    2    0    0

  -20  -13   -5    7    0    0    0    0

    2   10    0    0   -1    0    0    0

    1    4   -1   -1    1    0    1    0

   -1    0   -2    0    0    0    0    0

    0    2    0    0    0    0    0    0

    0    0    0    0    0    1    0    0

From a compression viewpoint, this is exciting. Most 8x8 pixel squares can be reduced to a bunch of low numbers and zeros -- easy to compress. This is how JPEG compresses data.

To recover the image, we multiply the stored, quantified values by the Q table to recover the set of frequency scalars. In this case, we get:

  -48 -144  112  -60    0  -21    0    0

    8   54    0  -22  -30   38    0    0

 -160 -117  -50   84    0    0    0    0

   24  110    0    0  -12    0    0    0

   17   60  -19  -12   12    0   12    0

  -21    0  -24    0    0    0    0    0

    0   24    0    0    0    0    0    0

    0    0    0    0    0   12    0    0

Now, this isn't exactly like the original data, but when converted from frequencies to pixels, it becomes "close enough." Of course, there are many different table values that generate the same results. So some of those values (including the zeros) may become non-zero. This means that more values will be dropped off the next time we resave and apply quantization tables. In fact, even 100% Q tables (where all values are "1") will yield a little loss because the transformation from pixels to frequencies requires fractional values and JPEG uses integers. (That's why 100% quality is really 99% quality.)

The net result is that multiple resaves will remove high frequency components from the 8x8 squares. What once were crisp edges are now blurs. However, the overall color will remain the same (approximately the average color for the entire 8x8 pixel square).

Finally, there is the 8x8 grid. Every 8x8 square is treated independently. A huge distortion in one square will not impact any neighboring squares. With one exception: subsampling. Depending on how the JPEG was saved, the chrominance components may use an 8x8, 8x16, 16x8, or a 16x16 grid. So let's say that the image uses a 16x16 grid. It means that no distortions in any 16x16 square will impact any adjacent 16x16 squares. They are all still independent.

Enough Math!

In theory, JPEGs will constantly get worse with each resave. In practice, JPEGs usually hit a local minima (where there are no more changes) after a few dozen resaves. For example, I found this relatively high-quality picture of Alyson Hannigan:


I resaved the image repeatedly at 99% quality. (Load, save at 99%, reload, resave at 99%, repeat.) At 99% quality, the changes stop after 11 resaves. (Since Q99 takes very tiny steps, it hits a local minima quickly.) Resaved files #11 through #500 all have the exact same sha1 checksum. At 75% quality, it stops after 54 resaves (saves #54 through #500 are identical). Here's the two images (and they really are just a little different):

Resave #500 at 75% quality	Resave #500 at 99% quality

In both cases, the differences from the original are minor. Her hair and sweater are barely less crisp than the original. (And the original that I started with isn't "original".) Let's compare this with frame #497 from the YouTube video:


Since there is no possible way an 8x8 JPEG square can become significantly darker or lighter with a just a resave, there has to be something else going on.

(For fun, I even asked Derek R. to repeat the resave experiment since he uploaded a little script to automate resaving. He wrote back: "I tried compressing the Hannigan pic 500 times, however, I couldn't produce the artifacts in your youtube video of the blocks gradually appearing. I tried a few compression ratios, and it would basically converge after several iterations.")

Thank You, MrGrundleFun

Originally, I was going to blog about how the youTube video was a lie. JPEG doesn't do that! However, I ended up digging a little deeper...

The key clue came from the YouTube video author, MrGrundlefun. In his video's text description, he wrote:

I took the original JPEG photo and opened it in Photoshop. Then I saved over itself as quality level 10 (out of 12). Then I closed the file and reopened it and did it again, 500 times. Each time I saved a copy and numbered them. Then I took every third picture and made this short movie out of them. If I used all 500, the movie would have dragged on too long, and the slow changes would be even harder to notice.

And there it is: Photoshop.

I repeated the experiment manually, using Photoshop. I lost count around 12 (doing it manually and the phone rang), but this is about 20 resaves:

With fewer than two dozen resaves, you can already see parts of the walls getting brighter and darker -- much more than the JPEG algorithm can account for.

Photoshop does some undocumented, proprietary magic to make high frequency areas appear a little sharper. (I think they are trying to mitigate loss from JPEG artifacts.) I've known about this for a few years and call it "rainbowing" -- it is a separation between the red and blue color channels that shows up during an error level analysis. (It's a tell-tale sign that an Adobe application, like Photoshop, was used.) Gimp does rainbowing a little; Photoshop does it a lot.

Now we have multiple JPEG resaves plus something other than JPEG happening between each resave. That "something other than JPEG" from Photoshop is enough to keep the image degradation from terminating after a dozen or more resaves.

Yes, repeatedly saving a JPEG makes the image worse. But repeatedly saving it with Photoshop makes it much worse.