Yesterday (today if you’re in the US), Sun released the latest version of its virtualisation solution, VirtualBox 3.1. Among speed improvements and other smaller features, the biggest news is that Virtualox 3.1 introduces something called teleportation: you can move running VMs between machines – servers or clients, different architectures, different host operating systems, it doesn’t matter to VirtualBox. Coincidentally, this reminded me of an idea I once had about moving running applications between machines.
Sun released VirtualBox 3.1 on November 30, and claims that it brings improved execution speed, improved network performance, and a 2D graphics acceleration feature for Windows virtual machines. Its biggest feature, however, is teleportation.
With this feature, you can move running virtual machines between different hosts. You can move them across different architectures, different host operating systems, and between servers and clients. this feature makes it possible to service your hardware without shutting down your VMs, as you can simply move them to another piece of hardware for the time being.
“The ability to teleport running virtual machines from one computer to another allows system administrators to perform essential maintenance with zero downtime of their IT systems,” said Jim McHugh, Sun’s vice president of data centre software marketing, “As a cross-platform hypervisor, VirtualBox allows customers to easily evaluate and deploy virtualized systems, using their existing x86 hardware, operating systems and skill sets.”
I’d like to make a big leap, as this new feature made me think of something that I sometimes think about when I’m mulling over the future of computing – more specifically, my perfect world of computing. One of the features I’d like to have is the ability to move running applications from one computer to the other – transparently, without having to shut them down or whatever. Let me illustrate with two scenarios.
You’re working on your home desktop. You’re trying to finish a document you’re working on, but you’re running a little late. How cool would it be if you could simply “move” the running application to your laptop, without having to worry about transferring the files you’re working on?
Or, the other way around. You’re on the train, working on a long email on your smartphone. You come home, and of course would prefer to finish the email on your desktop, with proper screen and keyboard. You just tap a button on your phone, and the mail application on your phone moves over to your desktop, where you can continue writing the email right away.
I’ve also been thinking about the user interface portion of this functionality, and that’s actually harder than you think. The straightforward option would be to have machines in range listed as icons, and you can just drag and drop a running application to the machine on which you want to continue using that application.
You could also think of more exotic ways of transferring applications – I thought of holding your phone next to your desktop’s or laptop’s display, and simply tap the display with your phone – the iPhone has an application called “Bump” which more or less works this way to transfer contacts, but it only works between iPhones.
It’s a big leap from VirtualBox to this hypothetical feature, but there you have it. I’m always surprised by the ingenuity of programmers, so I’m pretty sure other people have thought of this stuff way before I have. Do any of you know of efforts that seek to implement functionality like this? I have come to expect that the open source community especially is usually three steps ahead of everyone else in exotic functionality like this.
Look up ubiquitous computing in scholarly articles and start from there. It’s not my field, but it’s a pretty developed one, and they answer alot of your questions there
I actually saw and worked on articles on the idea behind the “teleportation” feature actually in school, 2 years ago. It’s fun seeing it “in the real world”!
Yes– It’s very nice right-clicking a node in my ESX cluster, selecting “Enter Maintenance Mode”, and watching all my running VM’s move to other nodes. Watching them move around in response to load is also nice.
Xen has also had this functionality for awhile, so I’m glad to see VirtualBox catching up.
I’m curious to see how “cross platform” it is– Under VMWare ESX, it’s annoying enough to move between two similar CPU’s with slightly different feature sets. If the running VM can’t tell that it’s 64-bit CPU just changed to a 32-bit CPU, then I would think that the VM isn’t capable of exploiting very much of the higher-end CPU’s.
This is where the “virtual CPU” comes into play. Not sure how it works in Xen/VMWare, but with KVM, you can select which type of CPU to expose to the guest.
For best compatibility, you can use a “qemu” CPU type, which is a lowest-common featureset CPU (everything that Intel and AMD support). Or you can be more specific on the CPU features to expose. Or you can just pass through the full CPU set of the host.
Thus, if you have all AMD CPUs in the VM farm, you can pass through the host CPU.
If you have AMD and Intel CPUs, you can use the qemu CPU for VM portability.
Moving a 32-bit guest from a 32-bit host to a 64-bit host would work. But going in reverse (64-bit guest to a 32-bit host) wouldn’t.
Gmail.
Bingo… often the best solution is one that already exists
While probably sufficient, that’s not quite there though. You can certainly write a draft email on machine and continue it on another, but you still have to make a conscious action to fetch the email from the draft folder.
Thom’s idea seems to have as much to do with preserving the entire application session in the state you left it. Fairly easy for a web app like GMail, somewhat harder for desktop apps…
Well yeah, I understand – but the likelihood that it will happen between disparate systems running different OSes (I know arguably, one could eventually run the same OS on a phone and desktop) in the next few years is pretty slim IMO.
In the meantime, I enjoy roaming Gmail chat discussions between multiple machines, and the nearly-instant availability of drafts from one machine to another.
Enter Google Wave protocol and it’s even more instantaneous compared to GMail as each participant of the wave is receiving live updates instantly.
You could conceivably manage a program’s running state as a Wave instead…
Looks like those rather nasty situations that never happen in real life.
Exactly. Think about this one, for instance.
I watch my television series on two computers – I have an HTPC in my bedroom as well as my living room. They both draw from the same storage space. OFten, however, I won’t be able to finish watching an episode on f. ex. my bedroom HTPC before I go to sleep – so I pause. Wouldn’t it be handy if instead of having to remember where I left off and scrub until that point on the other computer, I could just drag the video player to the other machine, preserving its state?
It sounds very handy to me, but it would probably require a heck of a lot of work, and I’m not sure if it is a desirable enough a feature.
Edited 2009-12-01 09:58 UTC
Theoretically it could work, but as the hardware capabilities will change underneath the player during playback you’ll probably have to configure it for the lowest common denominator and the limitations of VirtualBox. That means no hardware accelerated playback, no multichannel audio or AC3/DTS passthrough, no LIRC, etc. To make those things work would properly, as you say, would require a heck of a lot of work.
A much easier thing to implement would instead be a backend/frontend split where a shared backend could have many frontends.
Uh, seems like it would be a lot simpler if you could just drag a “bookmark” from one machine to the other… afterall, that’s all you need.
So far, you’re making up some pretty simple scenarios to support the notion of moving a virtualized app across multiple machines… if both machines contain the same software already, all you need to drag is the data in its current state. That data could contain the current settings/state of the app and it could pick up where it left off.
I’m glad you threw in the case of “shared storage” here, because that would have been my main sticking point on any concept of dragging a running app to another machine – what if the resources are missing. Every app has different needs, what if a DVD you’re playing from is in the desktop and you want to continue watching from the laptop…
I think it’s a solution looking for a problem, personally. If anything, I think you should be pushing seemless movement of application state from one machine to another. It seems nobody really focuses on this concept these days, because it’s assumed you’ll finish a given task on the machine you’re currently using.
I suspect a machine without any local storage is the most likely to support such a notion in the future, so look for it eventually on ChromeOS
There was actually a research OS out of Spain called Plan B that could do just as you described. I think it was based off of Plan 9. There were some videos floating around that were very impressive.
-Mike
You want X11 with XMove. Start an X app on PC1. Then XMove it to PC2 and carry on. Then XMove it to PC3. And then back again.
Granted, the actual execution of the app always happens on PC1. But the display, which is all you really need to worry about, occurs on PC2, then PC, etc.
All it takes is a network connection. Works quite nicely. And if you add in NX for better network response, it can even work across DSL and dial-up links (although not for full-motion video at those speeds).
That’s what IMAP is for! ISP’s simply use POP3 because they don’t want to store everybody’s email – which is fair I guess. But IMAP so a SO MUCH BETTER protocol for email. Mail is stored remotely including folders and filters. The IMAP server notifies you about new mail, you don’t have to keep polling it ever 10 minutes, etc…
I move back and forth between my mobile phone, home desktop and work desktop. I personal emails are all store remotely via IMAP and is always accessible from any device, even my web mail frontend reads from the IMAP server.
But does your webmail frontend automatically save your draft email progress as you’re writing it so you can go to any other machine and resume where you left off?
It’s pretty handy this way in case you accidentally close your browser while writing an email, or the machine you’re on dies (read: laptop on battery) – you just go to another machine and fire up gmail and there’s your draft.
I’ve yet to come across one that didn’t. Although I’ve only used a handful of webmail apps (Squirrelmail, Horde/IMP, Zimbra, some horrible perl one).
This doesn’t work. Teleportation requires the VM to be stored in shared storage. Hence, if you teleport your VM to your laptop, you won’t be able to use it while on the train since the VM will not be stored on your local disk but on the network (and I doubt you can access your network from the train).
This feature is called “live migration” in Xen and KVM/Qemu and vMotion in VMware. Moving a virtual machine’s state between different systems is comparatively easy but doing it without pausing the VM for a long time is tricky.
Typically it works like this: you monitor what memory that VM is changing. Then you copy all the VM’s memory to the destination machine. Then you check what’s changed and recopy that – this will be quicker, because hopefully most of the memory has not changed. Then you check what’s changed – again, hopefully less – and just copy that. You do this for a while and then you briefly pause the VM, copy the last of the state whilst it’s stopped, then start the copy on the remote machine. The VM does have to be stopped briefly but it can be an imperceptible delay. Once the process is finished you switch off the extra memory monitoring, so that you can get full performance back.
Trouble with this is that it’s not instant – you have to wait for a while after requesting the live migration. You can make it go faster by stopping the original VM right away and starting the copy running on the remote system before all the memory has been moved. Each time the remote copy tries to access memory that hasn’t been moved yet, it has to wait. Although this looks instant to the user it still takes time and hurts the VM’s performance; plus, if *either* machine crashes during the migration then the VM dies, which is a worse failure mode.
If you wanted really low latencies for the operation to complete you’d want to run the copying phase *all the time* so that very little needs to be done when the operator asks for a migration. VMware has a product that does this, Xen has integrated some support for it and there is work in progress to support this with KVM. They’re focusing on high availability concerns but I’m looking forward to being able to do my work in a VM on my desktop which continually syncs with my laptop, then just hitting a “move” button and 5 seconds later being able to walk off with the work VM running on the laptop.
Side note: it would be very interesting to do this sort of thing with applications, as Thom describes. It’s doable up to a point but believe it or not, that’s considered quite a bit harder than migrating a whole virtual machine. Complex as a whole x86 machine is, most OS APIs are far more complicated still. Whatsmore, the state involved is also less well documented and – given people keep extending their OSes – less stable over time. There has been some work on migrating limited kinds of processes on various OSes, it’s just that it’s really really hard to solve generally for a commodity OS.
Thanks for the thorough comment!
I’d say app virtualization might be more easily doable if it ran in a VM itself as with Java or the .Net CLI.
A modified Mono VM for example could keep track of an app’s state from the moment it’s started. As long as the app is managed the situation would be very similar to that of an OS VM.
Of course the challenge would be that apps need to support moving from a desktop PC to a cell phone. That would mean more work for a developer in order to handle different form factors… Cell phones might be getting better in term of screen resolution, they still require an entirely different UI.
People seem to be forgetting a couple of important factors with app teleportation:
1/ External references:
If an app is expecting a specific file / directory structure. For example:
* Photoshop / PSP with it’s dozens of filters, FX and masks held as plug in files.
* Or sound editing software with it’s dozens of VST(i) audio plug ins.
You’d have to copy the GFX plug ins / VST folder (depending on the example) along with the app – which could violate copyright law (in terms of VST(i) plug ins, a lot are worth hundreds of £££ per softsynth / effect bank) and have several hundreds of MB of data to instantly “teleport”.
This simply is not doable on either a legal nor technical point of view (legally you can’t have several copies of active software so, from a technical perspective, you’d have to move the data from one platform to another each time you “teleport”
Another issue is:
2/ software that expects specific hardware:
* CD burners expecting a CD write,
* BACS software expecting a smart card & reader,
* video conferencing expecting a mic and (web)camera…..
and so on.
So in short, I think the only workable applications for teleportation are for kind of apps that’s (for the most part) already available via the interweb cloud (webmail, Google Docs, etc).
Thus we end up back at a bigger question: how much do you trust organisations like Google with your personal data?
Teleporting whole OSs doesn’t negate hardware dependencies, but it does resolve point 1/ of this (now lengthy) post.
[edit]
Thinking about it some more: X tunnelling through SSH is probably the closest we have to Thoms dream on current technology.
However I’m not sure how easy it is to transfer an active application from one SSH -X session to another.
Edited 2009-12-01 10:58 UTC
About a year ago, I wrote a rather interesting bit of code for a company I was working at. I’m trying to get the IP for that released, so that I can open the code.
What did it do?
It was a self-discovering p2p network for distributed processing, that would move -threads- from one machine to another, completely transparently, preserving the state, balancing the load across the cluster.
And it could run nicely in the background on desktop PC’s across a corporate network. For a company that makes money doing data processing, being able to run a background daemon that makes use of the un-tapped power in desktop PC’s (thousands of employees), the potential there was incredible.
The prototypes were completely functioning. I had a pretty easy to use library that was disgustingly easy to use. You just wrote a thread, and a factory to create new work (new threads), and let the rest happen automagically.
Granted, it’s not virtualization, but it was working gloriously on massive ETL’s during system migrations.
There was software like this called Mosix. Is your project similar to what it did?
Sort-of, except mine was written in java, would migrate java threads, and was platform independent and agnostic.
Yes, it could run platform specific applications, shell scripts, or native code through wrappers.
But the idea was that any machine in the corporate network that had a 1.5 JRE (could have been backported to 1.4…) would be a potential participant in the ‘cluster’.
And yes, the cluster (like Mosix) was self-teaching and learning, meaning no staging of files binaries, etc. Everything would ‘just work’.
It’s awesome to see something like that actually work, having multiple systems contributing to the execution and success of a batch of threads that make up a ‘job’.
KDE4.4 is coming with remote plasmoids, so that you can have something like a music plasmoid accessible from both your desktop and phone. But that’s a long way from what Thom is describing, which is actually migrating entire applications from one device to another.
Realistically, the only way to get that to work is to have the applications running inside some sort of sandbox. Otherwise, you have to figure out how to copy the OS state from one machine to another, and there isn’t even any guarantee that would be possible (different CPU architecture, different OS, etc.). So you have to have some common framework running everywhere that contains all the application state for you. Copying a VM over is relatively easy, because it’s essentially just copying over the state of a single application into another identical instance running somewhere else. The tricky bit is getting that to work on generic apps. It might be possible to use the .NET or Java VMs as a sandbox, I’m guessing they could be modified to work this way relatively simply.
The easiest way of achieving this kind of interoperability probably goes back to the past, and not a future tech. Just have a mainframe somewhere, which serves the applications you are using to whatever screen you currently happen to be looking at.
Edited 2009-12-01 04:26 UTC
As you say the remote Plasmoid stuff is like what Thom describes, but it’s far from new. Either as idea or implementation.
Saw this years ago with QNX and the Neutrino(?) GUI, wher you could take a application and move it from one computer to another.
X11’s remote display capabilities gives you something similar. There’s even a handy Xmove utility for it. The app continues to run on the original computer, but the display moves over to another computer. Kind of fun to do on a LAN, where you can “throw” applications from your computer to a co-workers, to another co-workers, and back to yours. All without losing anything.
However, that does require X11 to be running on all the computers, and does require an active network connection to the “host” computer.
Everyone just needs to configure an NX server, and use that for everything.
Just “suspend” your NX session on the work computer, start it again on the home computer, and repeat as needed. 
Anyone remembers IBM java agents? They were supposed to migrate (and they did) from one VM to another. This isn’t new at all.
You’ve described the two scenarios that initially got me using remote desktop – my laptop has effectively acted as a dumb terminal for the last few years, running RDP connected to my desktop.
Granted, it’s not as seamless as what you described – things like seamlessrdp help, but there’s still a fair bit of setup & “geek-fu” required (ditto for remote X). What you’re describing sounds a lot like Citrix, but obviously that’s out of the reach of individual consumers.
It seems that the easiest solution, at least with current technology, would be an approach where you have a server in your home that serves applications Citrix-style. As long as you had persistent sessions, you could just run your applications off the server (from within the house or externally).
Personally, I find that approach much more palatable than the everything-in-the-cloud/browsers-as-the-universal-UI-for-everything approach. It also opens the door for less unnecessary duplication of computing power within the home, something I’ve been thinking about since the last home automation article here.
I agree. VNC was originally conceived to do this — keep a user’s session (and data) open on a server while the user roamed from room to room, PC to PC. It works great in a local/LAN environment, though maybe wastes some local PC power if the PCs are full-blown desktops.
The big drawback is the requirement for full-time network connectivity, which might be an issue while traveling and such. But I can live with having to copy my files in that case. Migrating several GB worth of OS, data and session every time I switch machines seems rather inefficient.
I want to set something up like this at home as well. I haven’t seen VNC or RDP support things like streaming video very well, but I guess that will come. A hybrid approach would be fine, too — remote control, remote HTTP proxying, or all local, as the needs dictate.
I threw together a quick review with screenshots… enjoy!
http://unixzen.com/virtualbox-teleportation
It seems that you cannot really move VM between different CPU architectures.
From the UserManual (page 105)
“The hosts must have fairly similar CPUs. While VirtualBox can simulate some
CPU features to a degree, this does not always work. Teleporting between Intel
and AMD CPUs will probably fail with an error message.”
…is if someone came up with a sort of viral framework for applications. Something that allows an app to spread itself across machines, maintaining the shared state on each, in a way similar to threading and clustering …
Mozilla Weave syncs your bookmarks, open tab information and passwords, etc. encrypted to a https-webserver and you can open use it with Firefox and the mobile-version: Fennic
Regarding the news, it’s good if it’s for free. VMWare tools are offered only in the paid product.
About Thom’s idea, it is better to be possible do the transportation of the data only, not the whole OS or the application itself. This may sign that you will need same apps in both lands, but it will be faster to migrate than whole OS/App/Data. Otherwise, using a remote session or putting the data in a server is better.
The transportation is helpful in a scenario in that you can not stop processes in order to do Hardware maintenance/upgrades. And the sync process may take a lot of time if these servers are doing massive data changes.
That said, what I really would love is something based in an environment like the Chrome proposal. I’m working in a document in the phone/handheld and when arriving home, I only put it in the cradle and the content is switched to be showed in a better screen, and I can continue to work with other input devices. If I loss the device the content is preserved because it is stored in the server.
Rgds,
VMware tools are also available in the free VMware Player. In fact, the free VMware Player 3.0 has VMware tools for a plethora of OSes: FreeBSD, Solaris, Windows, Linux.
Sorry, I would say
“The tools for VM migration between physical servers” are paid. Such feature is not included in that they call “VMWare Tools” – these are for better access to VMs, not for administer them.
Rgds.
You store the actual VM image on shared storage (NFS, CIFS, iSCSI, etc). It’s only the in-use memory that is transported from one host to the other.
Remote X11, rdesktop, VNC, and NX allow you to do this already, although not as seamlessly as some would like.
And what if I need to change/migrate the area in that such VM image is stored ? Is there a way to move it having the VM running ? – this is my understanding of “VM transportation”, not only the instance, but the image too.
Well, I was thinking in something that changes the presentation itself – like when a web page knows if you are using a browser in a desktop or in a mobile device. Doing a VNC/RDesktop of a desktop sized screen in the mobile device will be a scrolling challenge.
Rgds,
Then you use storage clustering and multipathing.
Would you really want to try and migrate a 10 GB OS+apps VM image? Even if you trimmed it down to the bare essentials, it would still be more than just moving the used RAM around.
Actually, not. I do prefer to have an outage of a couple of hours and make a clean migration.
My question was to understand the process itself. You did me a good solution. Thanks.
This could be an interesting approach to application installation and management, in an app bundle sort of way. Let’s see.
Step one, create an OS image for your virtual machine configured in a baseline kind of way.
Step two, invent a way to describe arbitrary alterations to the base system that configure it as required for a particular application. Ideally this would be as independent as possible from the state of your baseline such that the image could be swapped for another one with e.g. security updates, a newer kernel, etc., without requiring changes to the alterations.
Step three, describe an application in a way which allows it to be installed atop of a system configured by step two.
Step three, compose the baseline image with the alterations and application at run/invocation time, start up a virtual machine with the resultant image and have it auto-launch the application in a kiosk-mode type of manner.
Step four, migrate the running virtual machine anywhere you like. The application is now also installed there.
Each application would thus be isolated from others and be completely independent of the underlying system. Upgrades would be a matter of swapping out the step 2 and 3 files. Since there would be a small number of baseline images, ideally one, actual disk space would not be horrific. Now, apps would take up far, far more RAM… but nothing is perfect.
Your idea could be a great help on my VirtualBox http://www.bestdissertation.com/“>writing .
I was hoping to see an official release supporting FreeBSD as a host system. People have been sucessful in compiling the opensource version but it does not support (if I recall correctly) USB mounting and device mapping. Also, there is no use of the GPU also in the opensource version right?
Hope the next version will support FreeBSD.
Seems that the new Teleportation feature requires the source and target machines to have access to shared storage (iSCSI, CIFS/NFS) where the contents of the Virtual Machine are stored.
I’m a little bit disappointed. While shared storage provides for the fastest migration of VMs I was expecting that Virtual Box could send the entire VM (if required) across the network (using rsync, for example).
Oh, that would be so painful.
Typical migration time for a VM under VMWare ESX, using shared storage, is under 2 minutes.
Having to rsync even a mere 20gb virtual disk would be incredibly slow, and at some point you’d have to freeze disk I/O for some amount of time. I suppose, in theory, you could snapshot the disk, and mirror the snapshot, while you stream the journal to both the local machine and the remote machine, but just thinking about it gives me a headache.
Wasn’t that the claim of Java some 20 years ago? Write once and run anywhere?
They need to improve the shared folder performance as it is quite unusable at the moment.
Checking out a svn repository and deleting it off on a shared folder will show what it is like, but I couldn’t even delete a folder with about only 50 files in it… ‘rm’ command just hung up.
Your vision of the future got me thinking…
Moving applications would be cool but what about copy-paste?
If instead of moving an entire application from pc to laptop/smartphone, i would really like to see copy-paste to work between devices.
Automagically send the clipboard via bluetooth or something to my devices. My phone could function as the clipboard server…