Moving WhatsApp from WM10 to Android/iOS

Today I made an attempt to move my WhatsApp messages from WM10 to an Android Phone.

If the journey is TL;DR; jump to the end of this blogpost for a download link and some remarks on limitations.

In it’s greater wisdom Facebook has decided that each phone should have a separate backup destination that prevents easy migration. It looks like this:


Operating System Backup Destination
Windows Mobile 10 OneDrive
Android gDrive
iOS iCloud

This besides the on-device backups off-course which are incomplete as they are encrypted and the keys to decrypt it are saved outside reach for normal not rooted phones.

So what to do. I tried some tools to unlock my Lumia 550 but especially the x5x models are hard to gain access to. Moving the local backups from WM10 to an Android phone also fails. WhatsApp will only allow to restore backups at first start, just after installation. The WM10 files lack the encryption suffix that tells WhatsApp what encryption method was use.

After an afternoon without much success I attempted a totally different attempt. I notices that WhatsApp web (and the ‘native’ desktop version) shows all messages on my desktop. This application however does not store anything local as far as I could detect. It seems to use the phone it’s linked to for message retrieval.

So I tried to cut/paste conversations. WhatsApp web (both the browser version and the ‘native’ application) however do not support the almost universal shortcut Ctrl-A (select all). Selecting with a mouse by dragging is also awkward as not all messages are loaded. What works is to scroll totally to the top of a conversation and click somewhere outside a text balloon. Then scroll down again and click after the last text balloon. The text should color selected and Ctrl-C works.

Pasting it reveals Unicode text that starts with a square bracketed timestamp and a contact name.

So I started writing a little C# application that saves the clipboard if it contains Unicode text starting with the ‘[‘ opening bracket and the first line contains a closing ‘]’ bracket as well. Other encodings seem to loose the emoticons in the text so are of limited use.

WhatsApp protocol

During searches I came across URL’s that can be used for ‘click to chat’. I started experimenting with these links but they all had the issue that the browser based version would popup and display a message if you want to switch to the desktop application.

To get rid of this phenomena I used some code I had coded last year that registers a custom protocol to launch an executable. So next up was to locate the desktop WhatsApp executable. In Windows its not that hard to find a window by its caption (‘WhatsApp’) and obtain the executable. So enough data to register the whatsapp:// protocol to launch WhatsApp. Some details had to be coded like elevating the application so the registry can be written. Luckily code I had available from previous projects.

Now starting a URL with a phone number and UrlEncoded message as query parameter nicely drops the text into the edit field of the WhatsApp desktop application without a browser intervening.

Last issue to solve was how to submit the message. Some googling led to code that simulated a keypress on another window. Combined with the size and position of the WhatsApp window and a small offset for the location of the send button in the bottom right corner all started to work properly.

image

Basically what the migration tool does is send all saved chat conversations from yourself to yourself. This results in the messages all being in a single conversation but searchable in WhatsApp. Timestamps of the messages themselves carry the date/time of migration. However the tool posts complete messages including the original timestamp and sender (so searchable).

Notes

Note: The migration tool works with the x64 bits Windows version only (download from the WhatsApp download section).

Note: Selecting and Copying conversations takes some practice. Sometimes the timestamps are missing. Then just click at slightly different spots.

Note: Editing the pasted files is possible with Notepad. Make sure the files stay UTF-16/UTF-16 LE and have Unix line-ends (LF)

Note: Media are not including (but these are often saved separately and not encrypted on an SD-Card and can be retrieved easily.

Migration work-flow

  • Link the desktop WhatsApp Web (x64) application to your old phone.
  • Select, copy and past the conversations one by one onto the migration tool (and save them with a name matching the conversation). To paste either activate the migration tool window and press Ctrl+V or right click and use the context menu.
  • You now have a backup of all chat messages on your computer.
  • Link the desktop WhatsApp Web application to your new phone.
  • Make sure WhatsApp is detected properly and the ‘whatsapp://’ protocol can be registered. The migration tool needs to restart elevated to register (so you need to press the button again when the elevation shield is no longer shown).
  • Enter your phone number (full international number without a leading + sign)
  • Press the Send Messages button and take some coffee (I had to add some delays between messages).
  • Optionally the migration tool can prepend the filename to the message for better searchability.
  • The two blue underlined labels open explorer windows to the WhatsApp installation folder and the folder where chats are saved when pasting.

TL;DR; section

Recap: this migration tool transfers messages from your old phone to a single searchable conversation on your new phone excluding any media files.

  • The WhatsApp Migration tool can be downloaded from bitbucket.
  • C# sources are available at this bitbucket git repository.
Posted in Uncategorized | Leave a comment

WdFilter.sys & boot loop

Today my computer decided to start a boot loop where it claimed that WdFilter.sys was missing or damaged.

The file looked ok when inspected from the command prompt option of the Windows Recovery Environment (WRE). However in this same environment the option to fix startup issues continued to fail.

Restoring Restore points in the same WRE failed as well. The restore went well, the result was the same boot loop again.

Next try was to try-out all start-up modes from bootlog, safe mode etc. All resulting in the same boot loop. Strangly enough did the option to skip early virus scanning (of which WdFilter.sys seems a part as ‘Wd’ in WdFilter.sys seems to stand for Windows Defender).

Finally, in a last attempt before a reset of Windows, I decided to make a bold move and just rename the WdFilter.sys (which resides in the ‘C:\Windows\System32\drivers\wd‘ folder) to WdFilter.old and re-run the system start up repair option in the WRE.

To my surprise: Windows suddenly restored a correct copy of the filter driver and Windows booted correctly.

No guarantee but as it’s easily reverted (renaming WdFilter.old back to WdFilter.sys) trying might not harm.

But as it worked in my case I felt I should share this method.

Note: Only drawback so-far is that the renamed file can’t be deleted from within Windows due to file protection.

Posted in Uncategorized | 1 Comment

Double precision issues

Due to some interaction with a Swagger related project and a simple yet fast and easy to use C# JSON library SimpleJSON I used in the code generated by my swagger parser and code generator, I stumbled upon some issues regarding storing integers as double. The SimpleJSON library removed the task for generating huge amounts of model classes to deserialize REST call results into.

SimpleJSON uses double as internal storage, as that is fine as long as integer numbers are 32 bits. The trouble and mind-blowing issues start when you have to use 64 bits integers. In my case the Int64 numbers are sometimes used as ID’s of things to fetch. So they have to be exact. In the following text I ignore the unsigned integral numbers but they exhibit the same issue.

A simple examination shows both int64 and double are 8 byte data structures, so where’s the problem?

Double.MaxValue   (1.7976931348623157E+308)

is way larger then

Int64.MaxValue   (9223372036854775807 or 9.223372036854775807E+18)

But problems arise in the proximity of Int64.MaxValue to be precise.

The coding first attempt was to use the following in C#:

Double d = Double.Parse(Int64.MaxValue.ToString());

At first glance it returns a strange and incorrect value of 9.223372036854776E18, which is almost correct the correct value of 9223372036854775807, except that it’s only 7 off the correct value and 2 digits are wrong.

Given the byte-wise size of 8 for a Double, this is understandable, it reserves 52 bits for the fraction, 11 bits for the exponent and 1 bit for the sign (See IEEE Standard 754 Floating Point Numbers).

An Int64 in comparisment has a 63 bit integral part and 1 sign bit. So it can never fit with full precision into the Double fraction. It’s not the byte size of the double that is the limit, but the precision that is less because the double also contains a exponential part.

Doing the same with an Int64, e.g, load a number to big to represent, like:

Int64.Parse(“9223372036854775808”)

throws a nice out of range error.

The cause in this case is clear: the input is larger then the type’s MaxValue. When using a Double, Int64.MaxValue is still magnitudes  smaller than Double.MaxValue, therefor not triggering the same out of range error.

Trying to go safer with:

Double.TryParse(Int64.MaxValue.ToString(), out Double d)

returned true (e.g. no problem during conversion) and the same value that was 7 off. expected was false as the conversion is not flawless.

Even stranger is trying to convert the Double d outcome to a string using:

Double.TryParse(Int64.MaxValue.ToString(), out Double d); d.ToString(“F0”)

returned  “9223372036854780000” instead of the expected value 9223372036854775807. Now it’s a whopping 5 digits off track.

These issues might occur whenever data is stored as tekst and not as binary values, Because in formats like json there is often no way to determine whether a  value is an integer or a floating point:

5

might be a Byte, Int16, Int32 or Int64 but also a Float and a Double.

5.0

on the other hand is clearly a floating point number so a Float of a Double. As Double is the largest of the two, it’s the safes choice, it will fit.

Even a blunt bit by bit copy (just use the Double’s 8 byte as storage) will probably fail as a Double has some bit patterns that signal special numbers like +/- Infinity and NaN or ‘Not a Number’ and might trigger exceptions. Both of these special numbers have their exponential part filled with all 1’s. (See IEEE Standard 754 Floating Point Numbers).

As can be seen above, taking a Double is most of the time (but not always) a safe choice.

So it this all a C# problem/issues? By far!

In Java:

System.out.println(Double.parseDouble(“” + Long.MAX_VALUE));
System.out.println(“” + Long.MAX_VALUE);

returned 9.223372036854776E18 instead of correct value of 9223372036854775807 (so 4 digits wrong, due to some rounding it seems),

In 64-bit Python 3.6:

float(sys.maxsize)
int(sys.maxsize)
print:

returns 9.223372036854776e+18 instead too of the correct value of 9223372036854775807 (so like java 4 digits off).

PS. An unrelated issue is that in C/C++ parsing strings with methods like strtof() into numbers usually stops at the first character that is not understood. One of the returned values of for example strtof() is the index where the parsing failed. So in case of a wrong decimal separator you might end up with only the integral part (so 5 instead of 5.5235).

Posted in Uncategorized | Leave a comment

WHS 2011 Client Backup Drive Full

This week I had that dreaded message for the second time. Probably due to modern GB games that patch themselves regularly and entering the Windows Insider program (so a new Windows 10 version every now and then).

The last time (quite desperate) I deleted one of recent the Data.4096.nn.dat files and did a repair. It worked but I just lost a lot of backups. So I wanted to avoid that at all cost.

What happens when the client backup drive starts filling up is that beyond a certain point the weekly cleanup task will at most only adjust the indexes and not shrink the actual cluster storage files (the Data.4096.nn.dat and Data.512.nn.dat). So even if you mark backups as to be deleted at the next cleanup it still does not free-up disk space. If the disk becomes even more filled-up, even the adjustment of the cluster indexes stops after a few backup-ed machines.

Yesterday I found a much simpler and better (and not destructive to start with)!

First and very important is not to make things worse, so do not forget to stop both backup services so no backups are added during this operation.

It turned out that using the build-in compression feature of NTFS (which happened to be enabled on my client backup drive, so probably by default), could free up the GB’s I needed to get things working again. After compressing around 64 of the smallest Data.4096.nn.dat my free space went up from 4GB to 25GB (around 6GB more that the largest file on the disk).

As my client backup drive is 2TB, I was quite happy that I did not have to compress all files.

After that it was a matter of marking old backups as ‘to be deleted at next cleanup’ and run the clean-up job. After the cleanup it’s best to revert the compression so you can do the trick again if needed.

You can apply the compress attribute by selecting a number of files and right-click them for the property dialog. There use the Advanced Button. It takes a while to compress so take some coffee or better a lunch in the meantime.

For command-line lovers, the command to look for is called compact.

Removing the compression is just a simple compact /u * command from within the Client Computer Backup directory located in ServerFolders on the Client Backup Drive.

Posted in Uncategorized | Leave a comment

Reindexing WHS 2011’s DNLA Server

Searching for this subject reveals a couple of links of which each has some issues. So I started combining code and testing in so I would be able to rebuild the index without restarting the server.

The reason for rebuilding is that the indexing seems to work on directory notifications and also indexes files that are moved out of the indexed folders. If that happens one will start seeing drive letters in the DNLA file lists.

This proved somewhat more difficult. The database called ‘CurrentDatabase_372.wmdb’ is located under the profile of a special user called ‘MediaStreamingAdmin’. The processes related to the DNLA server also run under this account. Basically these are the services whsmss (Windows Server Media Streaming and HomeGroup Service) and WMPNetworkSvc (Windows Media Player Network Sharing Service).

Both services needs to be stopped before an attempt to delete the database can be made.

This however still fails when the server has run for a while and midnight has passed. The reason is that two other processes are started under the same account and accessing the same database (most solutions ignore this and ask to reboot the server before triggering a reindex).

These two processes are WMPAxHost .exe and WMPlayer.exe. WMPAxHost seem to control the WMPlayer.exe process and restart it when terminated. The purpose of these two processes seems to be updating the metadata of the media files with internet based metadata. This is probably the same feature WMPlayer offers when started interactively.

Terminating these processes will not be a problem as they are restarted next midnight. It’s off-course obvious that WMPAxHost has to be terminated before WMPlayer. To kill these processes some force has to be applied (hence the /f switches of the taskkill statements).

During the search for a solution I also came a way to disable and enable the Media Sharing with PowerShell commands. I have not tested if the batch file runs without these two lines as I find it more elegant to disable the Media Sharing feature during modification.

The last trick used is to rename the database file (which is under Windows NT or later allowed on files that are open). So even if final delete of the renamed database in the script fails, next reboot would create a new database.

The complete script looks like:

cd /d c:\program files\windows server\bin

wsspowershell.exe set-wssmediaserverenabled “-enable 0”

net stop whsmss
net stop WMPNetworkSvc

taskkill /f /im wmpaxhost.exe
taskkill /f /im wmplayer.exe

ren “C:\Users\MediaStreamingAdmin\AppData\Local\Microsoft\Media Player\CurrentDatabase_372.wmdb” *.old

net start WMPNetworkSvc
net start whsmss

wsspowershell.exe set-wssmediaserverenabled “-enable 1”

del “C:\Users\MediaStreamingAdmin\AppData\Local\Microsoft\Media Player\CurrentDatabase_372.old”

Posted in Windows | Leave a comment

Using a Denver AC-5000W with Windows (or OS X)

Some weeks ago, the Denver AC-5000W action camera’s where for sale for around € 50. So a lot cheaper than a GoPro and thus nice for testing. As it comes with a underwater housing up capable of withstanding water pressure up to 40m of depth, it’s useful with our scuba diving hobby without spending to much (one could always buy a GoPro later).

But this blog post is not about scuba diving or GoPro versus Denver, but about getting the stuff out of the camera (preferably by Wi-Fi without opening the case). The camera supports Wi-Fi by advertising itself as a Wi-Fi hotspot with a security key ‘1234567890’.

As either the Wi-Fi connection/feature or the mobile iOS/Android software is unstable (I was not able to download all photo’s with either of them and Android was way better in it then iOS), I wanted to know how to get the photos and videos off the camera using a PC.

First I though to disassemble the Android APK file but it proved a bit hard to download this file on a PC (I needed to enter my username/password and device code in a piece of unknown software). But it was not necessary to do this at all.

Then I just tried to connect with a browser to the gateway address (192.168.1.1) of the hotspot the Denver advertises (without luck). Normally this type of devices tend to expose a embedded webserver (like the average Wi-Fi router).

Ping this IP address however worked.

To be able to see a bit more of what goes on, I started with using telnet (so see if a connection was able at all). I tried ‘telnet 192.168.1.1 http’ so a webserver, again without luck. The second try was way better, ‘telnet 192.168.1.1 ftp’ because I got a nice welcome message and a prompt for a username.

Next was finding the username and password for this embedded ftp server. First I tried ‘admin’ and as password ‘1234567890’ assuming the programmers did not want to make it that hard. No luck.

Then with a little luck I tried good old ‘root’ as username and once again ‘12345679890’ as password and to my surprise I was in.

The camera shows a simple camera alike SD layout of a root folder DCIM with subdirectories for photos and videos and a additional one for events (no clue yet what that’s for, maybe it’s used for the feature to look at the live camera picture with a mobile device).

So the directory structure is a simple

DCIM  
  100EVENT
  100IMAGE
  100VIDEO

With a decent FTP client like FileZilla it’s very easy to transfer all photos and videos to a Windows PC or Apple Mac.

So just put the camera in Wi-Fi mode, connect to the ‘DENVER AC-5000W’ hotspot using ‘1234567890’ as security key.

Then setup a ftp connection to ‘192.168.1.1’ with a normal plain text username and password (‘root’ and ‘1234567890’) and start transferring your photos and videos.

Transfers run most smooth if you set your ftp client software to a single (one) transfer at a time.

Posted in Uncategorized | 82 Comments

WP8 LongListSelector and not correctly updating ContextMenu’s

These last days I have been working on a simple WP8 app that uses TvDb.com to keep track of the next/upcoming series episode to watch.

I made extensively use of the LongListSelector combined with a ContextMenu from the WP8 Toolkit found at CodePlex. I want to be able to short tap (navigate) and long tap (context menu). The DataContext supplied is a Dictionary hence the Key, Value and  KeyValuePAir stuff present in the code.

For the xaml I used code like this to make sure my C# code would be able to know which episode to mark as watched when a user long taps a list item (note: I removed all non essential attributes)

   1: <phone:PivotItem Header="upcoming">

   2: <phone:LongListSelector ItemsSource="{Binding UpcomingEpisodes}" >

   3:     <phone:LongListSelector.ItemTemplate>

   4:     <DataTemplate>

   5:         <StackPanel Tag="{Binding Value.Id}" Tap="Upcoming_Tap">

   6:         <toolkit:ContextMenuService.ContextMenu>

   7:             <toolkit:ContextMenu DataContext="{Binding Value.Id}" >

   8:             <toolkit:MenuItem Header="mark as watched" Click="UpcomingWatched_Click"/>

   9:             </toolkit:ContextMenu>

  10:         </toolkit:ContextMenuService.ContextMenu>

  11:         <TextBlock Text="{Binding Key.SeriesName}" />

  12:         <StackPanel Orientation="Horizontal">

  13:             <TextBlock Text="{Binding Value.EpisodeAndSeason}" />

  14:             <TextBlock Text="{Binding Value.EpisodeName}" />

  15:         </StackPanel>

  16:         </StackPanel>

  17:     </DataTemplate>

  18:     </phone:LongListSelector.ItemTemplate>

  19: </phone:LongListSelector>

note: I removed all non essential attributes.

The C# code is quite simple:

a) For the short tap I use:

   1: private void Upcoming_Tap(object sender, System.Windows.Input.GestureEventArgs e)

   2: {

   3:     if (sender is FrameworkElement && (sender as FrameworkElement).Tag != null)

   4:     {

   5:     Int32 id = Int32.Parse((sender as FrameworkElement).Tag.ToString());

   6:

   7:     // etc

   8:     }

   9: }

 

b) For the long tap I use:

   1: private void UpcomingWatched_Click(object sender, RoutedEventArgs e)

   2: {

   3:     if (sender is FrameworkElement && (sender as FrameworkElement).DataContext != null)

   4:     {

   5:         KeyValuePair<Serie, Episode> dc = (KeyValuePair<Serie, Episode>)((sender as FrameworkElement).DataContext);

   6:

   7:         //etc

   8:     }

   9: }

note: My DataContext is a KeyValuePair so I need to do some typecasting here.

The problem is that after marking a couple of episodes as read, the DataContext of the ContextMenu is not update correctly anymore and I keep marking things watched  I do not see in my LongListSelector.

After using Google for two days and found a ‘çomplex’ workaround I did not  get working at the one following links ‘we-secretly-have-changed’ or ‘dlaa’ I stumbled across an article at codeproject that led to the solution (I did not get the codeproject code to work in my project but searching for it at msdn did).

I modified my code a tiny bit at three places.

a) I added a

x:name=”UpcomingItem”

attribute to the topmost StackPanel element that defines an LongListSelector Item.

b) I changed the binding of the ContextMenu from”

{binding Value.Id}

into

{Binding ElementName=UpcomingItem},

effectively binding the ContextMenu to it’s parent StackPanel named UpcomingItem (so NOT to it’s DataContext anymore).

   1: <phone:PivotItem Header="upcoming">

   2: <phone:LongListSelector ItemsSource="{Binding UpcomingEpisodes}" >

   3:     <phone:LongListSelector.ItemTemplate>

   4:     <DataTemplate>

   5:         <StackPanel Tag="{Binding Value.Id}" Tap="Upcoming_Tap" x:Name="UpcomingItem">

   6:         <toolkit:ContextMenuService.ContextMenu>

   7:             <toolkit:ContextMenu  DataContext="{Binding ElementName=UpcomingItem}" >

   8:             <toolkit:MenuItem Header="mark as watched" Click="UpcomingWatched_Click"/>

   9:             </toolkit:ContextMenu>

  10:         </toolkit:ContextMenuService.ContextMenu>

  11:         <TextBlock Text="{Binding Key.SeriesName}" />

  12:         <StackPanel Orientation="Horizontal">

  13:             <TextBlock Text="{Binding Value.EpisodeAndSeason}" />

  14:             <TextBlock Text="{Binding Value.EpisodeName}" />

  15:         </StackPanel>

  16:         </StackPanel>

  17:     </DataTemplate>

  18:     </phone:LongListSelector.ItemTemplate>

  19: </phone:LongListSelector>

note: the phone:PivotItem has nothing to do with the problem described in this post.

c) Finally in the C# code I had to modify the retrieval of the DataContext dc variable as the (sender as FrameworkElement).DataContext is now  a StackPanel object instead of the KeyValuePair of the  original code.

So I changed the line 5 of the C# code piece above to read:

   1: StackPanel sp = (StackPanel)(sender as FrameworkElement).DataContext;

   2: KeyValuePair<Serie, Episode> dc = (KeyValuePair<Serie, Episode>)(sp.DataContext);

Finally the ContextMenu nicely works on the LongListSelector Item when long tapped, even when the underlying DataSource is updated.

Posted in C#, Programming, Windows | Leave a comment