The iPad Backup that took a long time…

This saturday I decided to create a full backup of my iPad. To cut a long story short: it took a long time. A very long time. A very very very very long time. In total the backup my iPad 16GB, which was filled with about 7GB of data, took a shocking:

18 hours, 22 minutes and 19 seconds!

Unbelievable. If you Google with Bing you can find a lot of users experiencing similar long backup times. As it turns out Apple has devised a backup mechanism, whereby backup duration is heavily dependent on the number of files it has to backup. Each file apparently takes several seconds to process, so backing up several thousand files can easily take hours.

I just had to investigate which apps caused these huge backup times. Analysis of the directory where iTunes had placed my backup showed that it basically contains pairs of two types of files: *.mdinfo and *.mddata. As these extensions imply, the *.mddata file contains the actual backup data and the related *.mdinfo file contains some meta data about that data file.

The *.mdinfo files are binary, but when opened with a text editor like NotePad they contain some readable elements. One of the more interesting things in this file is a text string beginning with the word "AppDomain". This looks like a unique identifier for an iPad app. I noticed that many files have the same AppDomain identifier in them, which leads to the conclusion that they all belong to the same app.

Based on this analysis I came up with the following scheme:

  1. Collect the AppDomain identifier for every *.mdinfo file in the iPad backup directory.
  2. Group all these files based on their AppDomain identifier.
  3. Sort these groupings, based on the total file count.

I wrote a quick C# console application for this and below are the results. First a little pie chart:

iPad App File Count

As you can see from the pie chart there are two apps that have claimed a really large number of files:

The winner (or better: loser) is the Wired app, the iPad version of Wired Magazine. It somehow needs 4947 files to operate. For the record: I installed the Wired app and did an in-app purchase of the issue featuring Toy Story 3. I didn’t buy any other issue.

The runner-up is the NU.nl app, which is an app that shows news content from the popular Dutch news website www.nu.nl. It’s a fantastic app, but I really haven’t got the faintest clue why it would need 3596 files to operate, since it should fetch its content live from the website.

The third place actually surprised me. For these files no AppDomain identifier could be identified. They did contain the string "MediaDomain", but I didn’t investigate in its meaning. However, since this group of files finished third, I’ve definitively needed to include them in my list.

Below you’ll find the full breakdown. Deleting this list’s top five apps from my iPad dramatically decreased backup times.

So, in the end I guess both app developers and Apple can learn something from this. App developers need to make sure their apps only use a limited number of files. And Apple needs to seriously fix their stupid backup mechanism.

AppDomain File Count
AppDomain-com.condenet.wiredmag 4947
AppDomain-nl.sanomadigital.nuhd 3596
*Unknown* 824
AppDomain-nl.veronica.veronicamagazine 659
AppDomain-se.illusionlabs.labyrinth2 504
AppDomain-com.nytimes.editorschoice 367
AppDomain-com.linkedin.LinkedIn 338
AppDomain-com.outfit7.talkingtomipad 314
AppDomain-com.vsallaccess.vsaaipad 240
AppDomain-com.tunaverse.iPadCinq 199
AppDomain-com.chillingo.cuttheropehd 154
AppDomain-com.isignz.aanbieding 121
AppDomain-uk.co.guardian.Eyewitness 113
AppDomain-nl.tvgids.app.ipad 86
AppDomain-org.pogi.dp 77
AppDomain-com.NewspaperDirect.PRiphone 73
AppDomain-com.atebits.Tweetie2 70
AppDomain-se.illusionlabs.touchgrind 64
AppDomain-com.concretesoftware.acestrafficpackipad 43
AppDomain-com.vitotechnology.StarWalkiPad 36
AppDomain-com.apple.Keynote 30
AppDomain-com.melodis.soundhound.free 24
AppDomain-com.dawikihoodcom 21
AppDomain-com.apple.Numbers 20
AppDomain-BibleReader 19
AppDomain-com.ikmultimediaus.amplitubepadfull 19
AppDomain-com.flipboard.flipboard 18
AppDomain-com.skype.skype 18
AppDomain-com.firemint.flightcontrolipad 17
AppDomain-com.davidme.soundpaper 16
AppDomain-com.goodiware.GoodReaderIPad 16
AppDomain-com.spotify.client 15
AppDomain-com.trautvetter.atomicbrowser 15
AppDomain-com.mobilepioneers.tvgidslite 13
AppDomain-com.smule.magicpiano 13
AppDomain-wefwef 13
AppDomain-com.chillingo.angrybirdsipad 12
AppDomain-com.evernote.iPhone.Evernote 12
AppDomain-com.amazon.Lassen 11
AppDomain-com.apple.iBooks 9
AppDomain-com.apple.Pages 9
AppDomain-com.booking.BookingApp 9
AppDomain-com.mobilepond.ipadphotogene 9
AppDomain-pocketinformantiPad 9
AppDomain-osfoora.hd 8
AppDomain-ch.graphicarteservice.newspaper.nrc 7
AppDomain-com.com2us.sliceit 7
AppDomain-com.ea.recklessracing.ipad.bv 7
AppDomain-com.fotopedia.heritage 7
AppDomain-com.layar.Layar 7
AppDomain-com.tmsoft.FartCushion 7
AppDomain-com.truphone.ipad 7
AppDomain-com.ebay.core.ipad 6
AppDomain-com.marcoarment.instapaperpro 6
AppDomain-com.teamviewer.teamviewerhd 6
AppDomain-com.ted.TED 6
AppDomain-com.aemotion.magworld 5
AppDomain-com.mythpeople.SparkleHD 5
AppDomain-com.natenai.glowhockey4ipad 5
AppDomain-com.sqrt3innovations.calculatorpad 5
AppDomain-nl.hyves.hyves 5
AppDomain-nl.telegraaf.krant.ipad 5
AppDomain-nl.vpro.3voor12.luisterpaal 5
AppDomain-at.co.hlw.itaprdpclient 4
AppDomain-ca.jubilo.ipadflickr 4
AppDomain-com.bryandev.GearedHD 4
AppDomain-com.Buienradar.BuienradarHD 4
AppDomain-com.gameprom.software.ThePinball 4
AppDomain-com.getdropbox.Dropbox 4
AppDomain-com.idealbinary.rumpel 4
AppDomain-com.imdb.imdb 4
AppDomain-com.jellyfilledstudios.AmzngDiceHD 4
AppDomain-com.laya.osiris.wikipanion.ipad 4
AppDomain-com.tarsin.Dilbert 4
AppDomain-com.thinkfun.rushhour 4
AppDomain-com.wolframalpha.wolframalpha 4
AppDomain-org.pogi.tigmatching 4
AppDomain-com.appshopper.app 3
AppDomain-com.europeandirectories.dtgipad 3
AppDomain-com.handypadsoft.deviceinfo 3
AppDomain-com.johnmoff.PocketPondHD 3
AppDomain-com.lucasarts.miseipad 3
AppDomain-com.vitotechnology.SolarWalk 3
AppDomain-com.wireddevelopments.rocketeeripad 3
AppDomain-de.meteogroup.WeatherProHD 3
AppDomain-fr.febo.multipong 3
AppDomain-com.appbc.touchable 2
AppDomain-nl.mangrove.Nozzman 2
AppDomain-com.tiogadesigns.bigcalculatorfree 1

Posted on October 25, 2010, in Uncategorized and tagged , . Bookmark the permalink. 11 Comments.

  1. This is another good reason to ignore Apple products. Don’t waste your time, wait for a good product from another manufacturer 🙂

  2. This is interesting.. nice apple pie..:)
    you said: “Deleting this list’s top five apps from my iPad dramatically decreased backup times.”
    My question is: how long did it take after deleting the top 5? Also, do you have a guess about about a #seconds / appdomain?

  3. My guess about the Unknown (media) domein would be the photo app with all the photos and videos.

  4. Leon, ik ben geinteresseerd in jouw c++ programma (run-only versie) om mijn eigen Ipad backup te controleren.
    Ik heb net mbv itunes 10.1 (net uitgegeven) of er iets verbeterd is, maar mijn PC cpu verbruikt is wel gedaald van 60-70% naar 25-30%, echter de doorlooptijd is niets verbeterd. Nog steeds een gigantische hoeveelheid mdinfo en mddata bestanden.
    mvg Jan Moen

  5. Did the same for my 7hr backup, see http://twitpic.com/39ei9y
    47% Wired, 13% Zinio.
    Perl script for counting this is on pastebin: http://pastebin.com/ka0LCtX9

  6. Post that C# console application. That would be useful.

  7. Several issues cause this problem. First there is a bad program design in NU HD. Every image shown in the news feed you select are downloaded to the iPad and stored in the Documents folder instead of the tmp folder of the app. Secondly they forget to cleanup this folder. My Nu.HD contained all the images it downloaded since I installed it (more than 6 months, > 50.000 files).The iTunes backup allways backups the Documents folder of every app on your iPad.
    I investigated the backup process of iTunes and found out that in my situation it toke 1.7 seconds to back up a single file (size did not matter). This slow process is caused by the fact that iTunes saves the backup status in a xml manifest. The data in this manifest is base64 encoded. Because you can abort the backup anytime, Apple updates this manifest on your Mac after every single file entry. On my Mac the file is 5 Mb. The size of the manifest + decoding it every time before writing it to disc causes this unacceptable slow backup.

  1. Pingback: Trage backups geanalyseerd - iPhone Forum - alles over de Apple iPhone, iPad en iPod touch

  2. Pingback: Mac, iPhone, iPad ve iPod ile ilgili uygulamalar, incelemeler, jailbreak ve son haberlerin paylaşıldığı teknoloji blogu.

  3. Pingback: outlet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: