VM Solution for consistence development environment

2

2disbetter

n00b
Joined
Jul 19, 2011
Messages
15
I have done a lot of digging and reading and have been using a solution for a few months now. I wanted to put it before you all and get some feedback on it, and perhaps if there might be a better way to go about it.

The premise is I have a computer setup with all the IDEs I use (VS 2013, Eclipse (w/ ADK)). It is configured to be small and streamlined (taking up a total of 30-ish GB in hdd space). The only things installed are those things necessary for developing in those two environments. (Compilers, SDKs, Git, etc.)

The issue I have had is my main desktop would be setup, my laptop would be set up, and the VNC computer I have on another subnet would be setup, but keeping settings and source code all sync'd between the three was a real pain in the butt.

So I moved this dev environment to a VM through VMWare workstation (Absolutely love workstation!). The idea then was to just sync the VM across the three computers and run it through workstation/fusion as necessary.

The advantage to this was no matter which computer I used it was the EXACT same environment. I didn't have to worry about whether I had sync'd the files before using it, etc. It was just a real convenience, and I really love it.

Now I have a little entry level NAS on my lowest tier network, which I was using to sync the VM, but it was slow, and time consuming as network speeds varied, and syncing to the VNC computer took especially long.

So I picked up a Transporter Sync with the intention of automating the process. The transporter is installed on the VNC computers subnet and keeps the 3 computers automatically in sync.

Initially this wasn't so nice, as the Transporter does not support block/delta updates and would upload whole 2gb files if anything changed within them. This took along time, and was not really what I was looking for.

Then someone reminded me about the snapshot feature of workstation, and magically my syncing times went from 30-40 minutes to 1-2 minutes. It was glorious.

The only drawback was the VNC computer is a mac mini and runs OS X (iOS development). It uses Fusion which does not support Workstation's snapshots. This meant the emergency dev environment it was suppose to provide would not be in sync even if all the necessary files where present.

For a while I just ignored it, and was just glad that the Mini was acting as yet another copy of those files in the event there was an issue with the transporter or it's associated hard drive. But time and time again I'd have an idea for a problem I was working on, or thought editing something I had already done, but only had my phone on me.

This lead me to look into finding a solution which would allow the VM to work on the Mac using the same snapshots the desktop and laptop were using. It's not ideal and is overly complicated but works well.

I'm running windows server 2012 r2 through fusion which is in turn running the dev environment through workstation. It's slow, but bearable.

So the questions are then:
1.) Is there a more efficient way to do what I'm doing?
2.) Is there a dropbox like private system other than the transporter which offers the same functionality but through a real NAS and not one limited by the USB 2.0 bandwidth?

You comments in general on the setup would be greatly appreciated. I've not explained all of the reason for doing it the way I am, just for the sake of brevity, but if you have a question I'd be happy to answer it.

Thanks for any all feedback.

2d
 
Shockey thanks for the link. Owncloud seems pretty awesome. I think the big issue here is bandwidth. As long as it can saturate a LAN connection it will work better than my current solution. (I realize part of that answer is going to be based on the NAS it's syncing with)

Do you have any experience with owncloud?

2d
 
dreamnid said:
First off - I'm not a Windows guy.

What kind of development are you doing? Are you doing web or desktop apps?

Throwing bittorrent sync sync out there. It doesn't quite do block level sync, but will transfer 4MB chunks. See http://forum.bittorrent.com/topic/17782-bittorrent-sync-faq-unofficial/
Click to expand...

Windows game development and Android app and game development.

I suppose the other question to this is using a VM in this manner the best way to have the same environment to work on projects on any computer I might use?

2d
 
Can you clarify this? "I'm running windows server 2012 R2 through fusion which is in turn running the dev environment through workstation. It's slow, but bearable."

My interpretation is that you're have a dev environment setup in Windows Server 2012 vm.

Everyone is going to have their own approach. I don't think yours is that great though, mainly because of performance, and that you're trying to auto sync binary files which will undoubtedly run into sync conflicts. Because it is binary, it is very difficult to fix.

The last part of the above quote is key, and another reason why I don't like your approach. You want development process to be as fast as possible, especially since you're using languages that has a compilation step. Development tends to be very iterative ( i.e. change a few lines, save, run unit tests, check results, repeat. ) so you want that iteration to be as short as possible as your time as a developer is very important. Running in a vm will not be that fast compared to running on bare-metal.

Setting up environments for your workstation and laptop does suck, but it should be more or less a one-time cost. If this is not the case, what and why is it changing a lot?

As for your VNC machine, couldn't you just VNC/remote desktop into your workstation, especially if it is only for quick one-offs?

You can setup something that will auto share your source code like Transporter, and since they are text files, the sync is much faster since the files are smaller, and it will be easier to handle conflicts. But I think the better solution is to use a distributed source control like git and have your computers sync to that.

Hope that makes sense. Again, everybody is going to be different.
 
dreamnid said:
Can you clarify this? "I'm running windows server 2012 R2 through fusion which is in turn running the dev environment through workstation. It's slow, but bearable."

My interpretation is that you're have a dev environment setup in Windows Server 2012 vm.
Click to expand...

Not quite. Because VMWare Fusion does not support snapshots made in VMWare Workstation, I'm actually running VMWare Workstation through the Fusion Server 2012 VM. So it is a VM within a VM.

The reasoning for this is purely because of the snapshot feature found in workstation. Using it greatly reduces sync time. (I'm at about 30-40 seconds per sync now, versus 30-70 minutes before)

dreamnid said:
Everyone is going to have their own approach. I don't think yours is that great though, mainly because of performance, and that you're trying to auto sync binary files which will undoubtedly run into sync conflicts. Because it is binary, it is very difficult to fix.
Click to expand...

Performance is a concern. I would much rather run the VM through Workstation directly on the host machine. Sadly I'm using a Mac Mini as the VNC host and really can't justify replacing it just because it would be more convienent, when this solution works. The slowdown is noticeable, but the only time the Dev VM would be used on this machine is in the case of sudo emergencies. (I thought of something to fix a compiler issue, or a bug, and don't have my laptop of desktop around. I can then use my phone and do it through a VNC connection.) It is not suppose to be used on a regular basis, and as such it's speed issues aren't really an issue in this case.

The last part of the above quote is key, and another reason why I don't like your approach. You want development process to be as fast as possible, especially since you're using languages that has a compilation step. Development tends to be very iterative ( i.e. change a few lines, save, run unit tests, check results, repeat. ) so you want that iteration to be as short as possible as your time as a developer is very important. Running in a vm will not be that fast compared to running on bare-metal.

dreamnid said:
Setting up environments for your workstation and laptop does suck, but it should be more or less a one-time cost. If this is not the case, what and why is it changing a lot?
Click to expand...

This is a good point. Honestly I'm just lazy. It takes a considerable amount of time to setup a system. Getting all of the includes, sdk's, etc. in place and compatible with links already found in existing projects is just too time consuming. Add on top of that the tendency to upgrade rather frequently and it is really just too much of a hassle.

I'm asking for opinions because I realize this is a rather unorthodox approach to this problem, but aside from the issue in using OS X, it works fantastic, and has made working a lot easier.

dreamnid said:
As for your VNC machine, couldn't you just VNC/remote desktop into your workstation, especially if it is only for quick one-offs?
Click to expand...

This is what I am doing with the nuance of actually using the dev VM inside of another VM.

dreamnid said:
You can setup something that will auto share your source code like Transporter, and since they are text files, the sync is much faster since the files are smaller, and it will be easier to handle conflicts. But I think the better solution is to use a distributed source control like git and have your computers sync to that.

Hope that makes sense. Again, everybody is going to be different.
Click to expand...

I use git locally on the Dev VM and I agree it would work to sync projects but I just don't have enough experience with it to use it sync a much larger state of a system than just document files.

I've tried goodsync as well, and it's a very good program too. It's issue is integration when you have more than 1 source.

Thanks for posting!

2d
 
I can't say i feel the same pains since im not a windows dev but perhaps i can show you some of the ideas i've kicked around.

So from what I understand you want to have a portable work station a vm perhaps but its clunky to pass around a VM disk right? If you use something like Vagrant or Ansible you can make portable dev enviroments that is defined by a simple file. The dev enviroments dont carry the full disk but instructions on how to set it up.

So you basically create this file slap it in your a repo in your NAS. Go to your computer you're about to develop on pull/clone that repo, go to the directory that contains the file and launch the VM. You can then use the VM from there to do development.

I can't gaurentee you'll be able to set everything up cause i dont dev with Windows but as php web developer I am able to launch a local enviroment that matches production.
 
Cottonwood00 said:
I can't say i feel the same pains since im not a windows dev but perhaps i can show you some of the ideas i've kicked around.

So from what I understand you want to have a portable work station a vm perhaps but its clunky to pass around a VM disk right? If you use something like Vagrant or Ansible you can make portable dev enviroments that is defined by a simple file. The dev enviroments dont carry the full disk but instructions on how to set it up.

So you basically create this file slap it in your a repo in your NAS. Go to your computer you're about to develop on pull/clone that repo, go to the directory that contains the file and launch the VM. You can then use the VM from there to do development.

I can't gaurentee you'll be able to set everything up cause i dont dev with Windows but as php web developer I am able to launch a local enviroment that matches production.
Click to expand...

Interesting idea. I'm going to look into them, but honestly it sound like this approach might be a little more complicated than the previous, and have the extra problem of not incorporating changes well. Every time the environment changes (I add an updated version of an SDK, etc.) I'll need to rebuild the file, which in turn would need to be re-added to all of the other computers I might use.

I've not heard of Vagrant or Ansible so I'll have to check them out. Thank you for your recommendation.

2d
 
I can't go into too much depth because im not a windows person but you would normally just maintain the Vagrant file(VM specs). So when you go to another machine, you update the your repo(Vagrant File) and say vagrant up --provision and it wil replicate the changes. no rebuilding just maintain the file to handle your specs.

There is also something called chef(chef server) which is used with vagrant which. So you can maintain a VM spec and then your VM's update themselves without manual intervention

Just something to think about, im by no means and expert but those are things i've been looking into so i thought i might share my ideas.
 
Interesting. I checked out vagrant. It's pretty cool stuff. It's basically a wrapper written in ruby that does the heavy lifting when setting a VM up. It seems to make a lot of sense if there are multiple environments you want to push out. (Ie: You have a team working on something) I'm just one person and I don't have that requirement. I'm not knocking vagrant because had I known about it before I did my own thing I would have probably looked into it then and maybe of used it as well.

Some of my reasons for not being so inclined with it now are:
-While it has vmware inclusion now, it was written primarily for use with VirtualBox. I'm hooked on VMWare after using VB for a while, and I don't think I could ever go back to VB. Probably a none issue as Vagrant does support VMWare.
-Since I've already configured and set up the VM specific to my needs, the extra bit of isolation for the programming environments and projects includes, etc. that vagrant offers seems like an unnecsary thing at this point. On top of that when I have to update the actual VM itself with windows updates, etc. this doesn't seem like it would be covered under vagrants purpose.

I looked at their website and also gleaned some info from this discussion here:
http://superuser.com/questions/584100/why-should-i-use-vagrant-instead-of-just-virtualbox

Once again thank you for your comments. Before I even began the virtual machine, I wondered what others in my situation used. I wasn't able to find anything in my attempts through google. It looks like vagrant is a serious contender in a lot of circles.

If I ever expand my operation, I think vagrant will be the first thing I check out for adding additional VMs with similar environs.

2d
 
Cool may i ask what you're final setup may seems like, are you going to try that btsync thing with the VM disk?
 
Well I have a working setup right now. The btsync looks interesting, but I have to be honest when I say that owncloud looks like a real winner. When I upgrade my NAS I'll seriously look into it. Currently since I already have the transporter sync, I'll just keep using the current method.

2d
 
I was going to suggest Vagrant coupled with Ansible/Puppet/Chef if he was doing web development in a *nix environment, but it's probably not worth the trouble setting it up for Windows ( don't have experience either, but doubt it will be as easy )

Sounds like you're happy with your setup. I'll still be careful of doing syncs of the vm hard drives, so I would do daily backups of the source code (using an outside git server should suffice) in case the files get corrupted.

Post again if you end up doing a Vagrant setup down the road. Curious if you can get that to work in Windows :)
 
You must log in or register to reply here.
Back
Top