TGIMBA .NET Core Upgrade – Part 8 – Continuous Integration / Continuous Deployment – Part 1 – Local Batch File

Git  Code Commit

This blog takes the manual process (see below at the bottom of this post) and automates it using a batch file.

The one exception is ftp’ing the distribution to my production server.  This process is so dependent on Visual Studio 2017 build tools (see caveats below), that I am pretty sure I will need to use Visual Studio Team Services (VSTS) to build this and rely on their deploy mechanism.  Working the ftp’ing process into the batch file is certainly possible (I believe), but redundant given this tight coupling. So I am leaving it til later.

In a previous attempt to automate the .NET 4.6.x The Globe In My Bucket Application (TGIMBA) website on Amazon using their Code Pipeline, I worked around the build requirement by simply copying the source files with all their required .dlls to the production server. Internet Information Systems (IIS) built it for me from there with the first web request. Trying to truly build the older version of the website on Amazon at that time would not have been possible.

In the spirit of full disclosure, this was a lot of trial and error 🙂 If you want to fast foward and see the batch file, please check out the git code link. The file is ‘tgimbaNetCoreBuild.bat’.

The main value in doing this was a trial run at what my build would look like in the cloud environment.  Having your own build server this days is just not worth the headaches.

I have to say that I was really impressed on how easy the Node JS API’s I have worked on in the past were to build via the command line (and then again on Amazon’s platform). It seems like everything was just there. I kinda felt like I had to hack this together (i.e. lots of dependencies that did not exactly work together easily).  I am hoping this is all a lot easier on VSTS.

The caveats:

  • This bat file must be run inside the Visual Studio 2017 Developer command prompt.
  • It makes extensive use of msbuild.
  • It uses the FolderProfile.pubxml file generated by the manual process.
  • It requires nuget.exe to do the restores (required for building a fresh check out from https://github.com/ehelin/TGIMBANetCore)
    • I downloaded nuget.exe (reference #16) and placed in build directory with .bat file
  • I had to do an npm install (which I thought was handled by the msbuild process). But, the deploy artifact requires the node_modules (i.e. Angular 4 TypeScript) and this is how I satisfied that required.
  • When doing an npm install via a Windows 10 .bat file, the command to use is ‘call npm install’ vs just ‘npm install’
    • If you don’t, your .bat file will stop after the install. I never saw this when working with npm installs previously on my local Macintosh or the Amazon cloud build tools.
  • I copy my local credential file (i.e. Credential.cs) into the distribution. Never check credentials into source control. Once I have my Continous Integration/Continous Deployment (CI/CD) cloud environment setup, I will most likely inject these via environment variables (or something).
  • Requires npm to be installed (kinda a no-brainer 🙂 )
  • I had wanted to keep all of the files local to my build directory, but the nuget restore seems to ignore my desired repository setting. I did some reading and it looks like this is possible, but it seems like you need to create nuget.config file and that was a little more work than I wanted to invest in this local file.
  • Running the Angular 4 tests seems to require spinning up a server to do so. Makes sense, but it does not exit upon completion. The karma help options do not show an easy way to accomplish this. The only way to do so that I have found so far is to kill the process. I will have to figure out how to do this in my cloud pipeline. If there is not an easy alternative, I will have to rethink how I am architecting this. I had hoped to have everything covered with unit tests (including the Angular 4 TypeScript code). If there is not an easy alternative, I may have to rely on integration tests (i.e. Selenium or something).  For now, the Angular 4 tests run at the end of the .bat file and have to be manually stopped.
  • mstest apparently is not meant for .net core (reference #17). Use dotnet test in the .csproj of your test project. I have 4. So, I will have to run four different ‘dotnet test’ calls. I would prefer to be able to ‘discover’ any new tests. But, this is better than not running any tests 🙂

The .bat file runs:

  • Sets a working directory path
  • Cleans up previous directories and recreates them
  • lcones the website from https://github.com/ehelin/TGIMBANetCore.git
  • Cleans the solution
  • Moves Credential.cs and FolderProfile.pubxml to their respective places in the source code
  • Restores the nuget packages
  • Runs an npm install
  • Builds the solution
  • Runs .NET Core tests
  • Publishes the .NET Core/Angular 4 deploy artifact
  • Copies previously install node_modules to the deploy directory
  • Runs Angular 4 tests

Running the process:

  • Directory prior to executing the build

prebuild

  • Directory after

postbuild

  • PublishOutput (what you deploy to your server…note that node_modules must be part of your deploy artifact.  If it is not, the Angular 4 portion of your site will not work.  The publish function (out of the box) does not appear to account for this.  I had to copy the node_modules over)

publishoutput

manual version ————————————————————
NOTE: release folder set inside C:\Users\…\source\repos\TGIMBANetCore\TgimbaNetCoreWeb\Properties\PublishProfiles\FolderProfile2.pubxml

NOTE: run from vs 2017 developer command prompt

  • cd C:\Users\…\source\repos\TGIMBANetCore\TgimbaNetCoreWeb
  • rmdir C:\tmp\tgimbaBuildRelease\node_modules /s /q -Force
  • rmdir C:\tmp\tgimbaBuildRelease /s /q -Force

NOTE: seems like I have to run this twice

  • rmdir C:\Users\…\source\repos\TGIMBANetCore\TgimbaNetCoreWeb\node_modules /s /q -Force
  • rmdir C:\Users\…\source\repos\TGIMBANetCore\TgimbaNetCoreWeb\node_modules /s /q -Force
  • msbuild C:\Users\…\source\repos\TGIMBANetCore\TgimbaNetCore.sln /t:Clean,Build
  • msbuild C:\Users\…\source\repos\TGIMBANetCore\TgimbaNetCore.sln
  • npm i npm@latest -g (sometimes required)
  • npm cache clear –force (sometimes required)
  • npm install (May note be required (part of msbuild…assuming no errors))
  • rmdir c:\tmp\tgimbaBuildRelease /s /q
  • mkdir c:\tmp\tgimbaBuildRelease
  • msbuild C:\Users\…\source\repos\TGIMBANetCore\TgimbaNetCoreWeb\TgimbaNetCoreWeb.csproj /p:DeployOnBuild=true /p:PublishProfile=FolderProfile2.pubxml /p:Configuration=Release
  • mkdir c:\tmp\tgimbaBuildRelease\node_modules
  • xcopy C:\Users\…\source\repos\TGIMBANetCore\TgimbaNetCoreWeb\node_modules c:\tmp\tgimbaBuildRelease\node_modules /E

References

  1. https://stackify.com/beyond-iis-logs-find-failed-iis-asp-net-requests/
  2. https://stackoverflow.com/questions/1965787/how-to-delete-files-subfolders-in-a-specific-directory-at-command-prompt-in-wind
  3. https://superuser.com/questions/206036/commmand-line-command-to-copy-entire-directory-including-directory-folder-to-a/206037
  4. https://stackoverflow.com/questions/45553401/npm-using-stale-package-data
  5. https://stackoverflow.com/questions/1752677/how-to-recursively-delete-an-entire-directory-with-powershell-2-0
  6. https://stackoverflow.com/questions/1965787/how-to-delete-files-subfolders-in-a-specific-directory-at-command-prompt-in-wind
  7. https://stackoverflow.com/questions/1965787/how-to-delete-files-subfolders-in-a-specific-directory-at-command-prompt-in-wind
  8. https://social.msdn.microsoft.com/Forums/vstudio/en-US/4fb20a03-bf73-472d-824f-4002c3db228f/forcing-a-target-to-clean-and-rebuild-using-msbuild?forum=msbuild
  9. https://stackoverflow.com/questions/4965507/msbuild-poutputdir-c-mydir-being-ignored
  10. https://stackoverflow.com/questions/21033801/checking-if-a-folder-exists-using-a-bat-file
  11. https://stackoverflow.com/questions/856573/concatenate-variables
  12. https://stackoverflow.com/questions/42305275/creating-a-bat-file-with-npm-install-command
  13. https://social.msdn.microsoft.com/Forums/vstudio/en-US/22200da6-b615-48ec-97fa-947f0a6af48b/how-to-restore-nuget-packages-when-using-msbuild-command-line?forum=msbuild
  14. https://gsferreira.com/archive/2014/08/automate-nuget-package-creation-using-grunt/
  15. http://www.visualstudioresources.com/2017/10/04/how-to-get-the-location-of-nuget-exe-used-by-visual-studio/
  16. https://www.nuget.org/downloads
  17. https://github.com/Microsoft/vstest/issues/1213
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s