Ping Results with CSV

This is a project that came out of a wish list from a coworker to monitor,  at a glance, if something is up or down as well as have a bit of a history to analyze for any slow or dropped pings, hence the graph.  The first phase (script) ping.ps1 for performing the ping tests is from an external source that my coworker found and then modified by the two of us to get what we want, the second phase of the process is GeneratePingCharts.ps1 written by me to generate the graphs from the output of the first script using RGraph which is an HTML 5 charts library.

GeneratePingCharts.ps1 output sampleping.ps1 output sample
Ping Results GraphsPing Results HTM

GeneratePingCharts.ps1

I am writing about this one first since I wrote it from scratch. The script calculates the ping response time averages for month, week, & day utilizing the data collected and saved in a CSV file and outputs the graphs and averages to an HTML file. For flexibility I allow the folder containing the CSV files, where the HTML should be outputted, and the location of the RGraph library files to be passed as parameters to the script.

The script starts by outputting the CSS utilized and then gathers the list of files containing the CSV extension from the folder path passed in.  For each file found a new job is started, with a maximum of 10 jobs running at a time, to process the CSV file.  The script waits for all 10 jobs to finish  and then removes the jobs prior to starting 10 more jobs.  This process continues until all files have been handled.

The line which starts the job by running the script block and passing the parameters is:

Start-Job $ProcessCSVScriptBlock -ArgumentList $file.FullName,$outputPath,$rGraphJS

The script block then initiates the processing of the CSV file by calling the Process-CSV function.  The CSV file is imported into a variable using the Import-CSV PowerShell cmdlet followed by each row being processed.  The sum of all the response times and number of response times collected is tracked to calculate the averages.  The response times are stored in a string with the format [a,b,c,….,z] where a-z is the response time value and will contain as many as there are stored in the CSV for the given time frame.  The only exception to this is month, which stores the average response time over $mInc points; otherwise, the graph would not draw do to the large quantity of points.  All this data is then stored into a custom object which is passed to the Generate-Chart function.

The Generate-Chart function stores the HTML for the graph page in a variable inserting the appropriate information stored in the $chartOut parameter in with the HTML string so that the:

  • Machine name and run time are displayed at the top
  • Averages are displayed in a table
  • Day, week, and month graphs contain the appropriate data points.

Worth mentioning is that the Get-My-Date function is used to convert the date stored in an unsupported format (MM_DD_YYYY) to a supported format (YYYY-MM-DD) so that within the Process-CSV function the date can be compared against today’s date allowing the script to know which graphs the ping result belong too.

<#
	.SYNOPSIS
		Calculates the ping response time averages for month, week, & day utilizing the data collected and saved in a CSV file by the script ping.ps1
		Creates a graph based on RGraph functionality and outputs all results to an HTML file.
	.PARAMETER csvFolderPath
		Where the CSVs are located and must all be in the same directory
	.PARAMETER outputPath
		Where the generated graphs should be saved
	.PARAMETER rGraphJS
		Where the JavaScript files are for RGraph used to generate the charts, relative to outputPath
	.EXAMPLE
		.\GeneratePingCharts.ps1 -csvFolderPath "C:\scripts\RESULTS" -outputPath "C:\Program Files (x86)\Lansweeper\Website\PingGraphs" -rGraphJS "../js"
#>
 
[CmdletBinding()]
 Param (
	[Parameter(Mandatory=$True)]
	[string] $csvFolderPath,
	[Parameter(Mandatory=$True)]
	[string] $outputPath,
	[Parameter(Mandatory=$True)]
	[string] $rGraphJS
)
 
$ProcessCSVScriptBlock = {
	param($fName, $op, $jsPath)
 
	<#
		Converts the date format used in the file to what Get-Date needs so we can do our comparisons
	#>
	function Get-My-Date ($date)
	{
		$parts = $date.Split('_')
 
		# year-month-day
		$df = $parts[2] + "-" + $parts[0] + "-" + $parts[1]
 
		return Get-Date $df
	}
 
	# Need to test
	function Generate-Chart
	{
		Param($chartOut,$op,$jsPath)
 
		$today = Get-Date
		$html = '<html>
				<head>
					<link rel="stylesheet" href="style.css" type="text/css" media="screen" />
 
					<script src="'+ $jsPath +'/RGraph.common.core.js" ></script>
					<script src="'+ $jsPath +'/RGraph.common.dynamic.js" ></script>
					<script src="'+ $jsPath +'/RGraph.common.tooltips.js" ></script>
					<script src="'+ $jsPath +'/RGraph.line.js" ></script>
					<script src="'+ $jsPath +'/jquery.min.js" ></script> 
					<!--[if lt IE 9]><script src="'+ $jsPath +'/excanvas.js"></script><![endif]-->
 
					<title>' + $chartOut.MachineName + '</title>
				</head>
				<body>
 
					<h3>' + $chartOut.MachineName + '</h3>
					<h5>Generated on ' + $today + '</h5>
					<table border="1">
						<tr>
							<th colspan="3">Response Time Averages (ms)</th>
						</tr>
						<tr>
							<th>Month</th>
							<th>Week</th>
							<th>Day</th>
						</tr>
						<tr>
							<td align="left">' + $chartOut.mAvg + '</td>
							<td align="center">' + $chartOut.wAvg + '</td>
							<td align="right">' + $chartOut.dAvg + '</td>
						</tr>
					</table>
					<hr>
					<h5>The Past Day</h5>
					<canvas id="Day" width="2000" height="300">[No canvas support]</canvas>
					<br/>
					<h5>The Past Week</h5>
					<canvas id="Week" width="2000" height="500">[No canvas support]</canvas>
					<h5>The Past Month</h5>
					<canvas id="Month" width="2000" height="500">[No canvas support]</canvas>
 
					<script>
						$(document).ready(function ()
						{
							var line = new RGraph.Line({
								id: ''Week'',
								data: ' + $chartOut.wData + ',
								options: {
									tooltips: ' + $chartOut.wDataToolTips + '
								}
							}).draw()
						})
					</script>
					<script>
						$(document).ready(function ()
						{
							var line = new RGraph.Line({
								id: ''Day'',
								data: ' + $chartOut.dData + ',
								options: {
									tooltips: ' + $chartOut.dDataToolTips + '
								}								
							}).draw()
						})
					</script>
					<script>
						$(document).ready(function ()
						{
							var line = new RGraph.Line({
								id: ''Month'',
								data: ' + $chartOut.mData + ',
								options: {
									tooltips: ' + $chartOut.mDataToolTips + '
								}	
							}).draw()
						})
					</script>
				</body>
				</html>'
		$html | Out-File -FilePath $($op + "\" +$chartOut.MachineName + ".html")
	}
 
	<#
		Reads a csv file in the appropriate format and calculates the last month, week, day response times (RT) as well as every date and response time.
 
		$file is the name of the file to process
	#>
	function Process-CSV
	{
		Param($file,$op,$jsPath)
 
		# Arrays for our chart data
		$mChartData = "["
		$wChartData = "["
		$dChartData = "["
		$mDataToolTips = "["
		$wDataToolTips = "["
		$dDataToolTips = "["
 
		$today = Get-Date
		$monthAgo = $today.AddMonths(-1)
		$weekAgo = $today.AddDays(-7)
		$mSum = 0
		$mPoints = 0
		$wSum = 0
		$wPoints = 0
		$dSum = 0
		$dPoints = 0
		$mIncSum = 0
		$mInc = 3 # How many points in month data to average to 1 point
 
		$csv = Import-CSV -Header MachineName,UpDown,RunDate,PingTime,Status,Timeout,TTL,RTTL,ReI,RT $file
 
		foreach($row in $csv)
		{
			# Array variable containing month, day, year
			$rDate = Get-My-Date $row.RunDate
 
			# Add down pings as empty, not included in sum/points for avg's
			if($row.UpDown -eq "down")
			{ #Need to figure out how to move the x-axis to allow negatives
				$mChartData += ","
				$wChartData += ","
				$dChartData += ","
				$mDataToolTips += "'" + $row.RunDate + ": " + $row.RT + "',"
				$wDataToolTips += "'" + $row.RunDate + ": " + $row.RT + "',"
				$dDataToolTips += "'" + $row.RunDate + ": " + $row.RT + "',"
			}
			else # Use the values in the file
			{
				# Handle month ago data
				if($monthAgo -lt $rDate)
				{
					$mSum += [int]$row.RT
					$mIncSum += [int]$row.RT
					$mPoints += 1
 
					if($mPoints % $mInc -eq 0)
					{
						$avg = ([double]$mIncSum / $mInc)
						$mChartData += [String]$avg + ","
						$mDataToolTips += "'" + $row.RunDate + ": " + [String]$avg + "',"
						$mIncSum = 0
					}
				}
 
				# Handle week ago data
				if($weekAgo -lt $rDate)
				{
					$wSum += [int]$row.RT
					$wPoints += 1
 
					$wChartData += $row.RT + ","
					$wDataToolTips += "'" + $row.RunDate + ": " + $row.RT + "',"
				}
 
				# Handle day ago data
				if(($today - $rDate).TotalHours -lt 24)
				{
					$dSum += [int]$row.RT
					$dPoints += 1
 
					$dChartData += $row.RT + ","
					$dDataToolTips += "'" + $row.RunDate + ": " + $row.RT + "',"
				}
			}
		}
 
		$mChartData = $mChartData.Trim(',') + "]"
		$wChartData = $wChartData.Trim(',') + "]"
		$dChartData = $dChartData.Trim(',') + "]"
		$mDataToolTips = $mDataToolTips.Trim(',') + "]"
		$wDataToolTips = $wDataToolTips.Trim(',') + "]"
		$dDataToolTips = $dDataToolTips.Trim(',') + "]"
 
		# Commented out wChartData & dChartData as mChartData contains all points and then can just use appropriate sections when generating the charts
		$chartData = New-Object System.Object
		$chartData | Add-Member -Type NoteProperty -Name mAvg -Value ([double]$mSum / $mPoints)
		$chartData | Add-Member -Type NoteProperty -Name wAvg -Value ([double]$wSum / $wPoints)
		$chartData | Add-Member -Type NoteProperty -Name dAvg -Value ([double]$dSum / $dPoints)
		$chartData | Add-Member -Type NoteProperty -Name mData -Value $mChartData
		$chartData | Add-Member -Type NoteProperty -Name wData -Value $wChartData
		$chartData | Add-Member -Type NoteProperty -Name dData -Value $dChartData
		$chartData | Add-Member -Type NoteProperty -Name mDataToolTips -Value $mDataToolTips
		$chartData | Add-Member -Type NoteProperty -Name wDataToolTips -Value $wDataToolTips
		$chartData | Add-Member -Type NoteProperty -Name dDataToolTips -Value $dDataToolTips
		$chartData | Add-Member -Type NoteProperty -Name MachineName -Value $csv[0].MachineName
 
		Generate-Chart $chartData $op $jsPath
	}
 
	# Do what you need to do
	Process-CSV $fName $op $jsPath
 
	# Just wait for a bit...
	Start-Sleep 5
}
 
# Entry point for script
#CSS file only needs to be done once so do it here
$css = 'body {
	font-family: Arial;
}
 
pre.code {
	padding: 5px;
	background-color: #eee;
	border: 2px dashed gray
}'
$css | Out-File -FilePath $($outputPath + "\style.css")
 
$csvFiles = Get-ChildItem $csvFolderPath -Filter *.csv
$count = 0 
Foreach($file in $csvFiles)
{
	$count += 1
	Start-Job $ProcessCSVScriptBlock -ArgumentList $file.FullName,$outputPath,$rGraphJS
 
	if($count % 10 -eq 0)
	{
		# Wait for 10 jobs to complete
		While(Get-Job -State "Running") { Start-Sleep 2 }
 
		# Display output from all jobs
		Get-Job | Receive-Job
 
		# Cleanup
		Remove-Job *
	}
}
 
Write-Host "Out of For waiting for last jobs"
# Wait for last jobs to complete
While(Get-Job -State "Running") { Start-Sleep 2 }
 
# Display output from all jobs
Get-Job | Receive-Job
 
# Cleanup
Remove-Job *

ping.ps1

The script reads a list of machine names listed one per line in a text file (named servers.txt below) and then loops through every machine and rights the results of the ping to a CSV file as well as the up results to a file with the appropriate HTML to append to the down HTML so that all the downs are displayed at the top.  The script also writes the machine name, state (up/down), run date, ping time, status code, timeout, time to live (ttl), response time to live (RTTL), reply inconsistency (ri), and response time (RT) to a CSV file for historical purposes as well as to feed the second script, detailed after the ping.ps1 code.  This script is presently being run as a scheduled task every 1 minute.

# Red = #FF0000
# Green = #00FF00
# Blue = #0000FF
# Cyan (blue and green) = #00FFFF
# Magenta (red and blue) = #FF00FF
# Yellow (red and green) = #FFFF00
 
#Win32_PingStatus class
#http://library.wmifun.net/cimv2/win32_pingstatus.html
#    11001 Buffer Too Small
#    11002 Destination Net Unreachable
#    11003 Destination Host Unreachable
#    11004 Destination Protocol Unreachable
#    11005 Destination Port Unreachable
#    11006 No Resources
#    11007 Bad Option
#    11008 Hardware Error
#    11009 Packet Too Big
#    11010 Request Timed Out
#    11011 Bad Request
#    11012 Bad Route
#    11013 TimeToLive Expired Transit
#    11014 TimeToLive Expired Reassembly
#    11015 Parameter Problem
#    11016 Source Quench
#    11017 Option Too Big
#    11018 Bad Destination
#    11032 Negotiating IPSEC
#    11050 General Failure 
#		is in use on other pages:	background-color  #DCDCDC
 
# <--------------- Start script ------------------------------------------------->
clear
#ipconfig /flushdns
$pingResults =("C:\Program Files (x86)\Lansweeper\Website\PingResults.HTM") # <-- you need to change this
$up =("C:\scripts\up.txt") # <-- you need to change this
$RunDate = (get-date).tostring("MM_dd_yyyy")
$PingTime = (Get-Date -format 'hh:mm')
$PingMachines = Gc "C:\scripts\servers.txt"
$n=(6)
#Write the preamble of the report
#clear-content -Path $pingResults
clear-content -Path $up
$htmlContent += "<head><meta http-equiv='refresh' content='15' ><p>"
$htmlContent += "<title> Ping Results </title>"
$htmlContent += "</head><body bgcolor='#DCDCDC'>"
$htmlContent += "<h3><p align='center'>Report Generated " + $RunDate + "
@ " + $PingTime + "</p></h3>"
 
$htmlContent += "<table border='1' align='center' style='width:50%'>"
 
ForEach($MachineName In $PingMachines)
{$PingStatus = Gwmi Win32_PingStatus -Filter "Address =
'$MachineName'"
Select-Object StatusCode
$status = ($PingStatus.StatusCode)
#Select-Object Timeout
$timeout = ($PingStatus.Timeout)
$ttl = ($PingStatus.TimeToLive)
$rttl = ($PingStatus.ResponseTimeToLive)
$ri = ($PingStatus.ReplyInconsistency)
$RT = ($PingStatus.ResponseTime)
If ($PingStatus.StatusCode -eq 0)
{
	Add-Content -Path $up ("<tr><pre><h6><td>"  + $MachineName +"</td><td><FONT color=#00FF00>`tUP</FONT></td><td><a href='./PingGraphs/" + $MachineName + ".html' target='_blank'>Graph</a></td></h1></pre></tr>")
}
Else
{
    $htmlContent += "<tr><pre><h6><td>" + $MachineName + "</td><td><FONT color =#FF0000>`tDOWN</FONT></td><td><a href='./PingGraphs/" + $MachineName + ".html' target='_blank'>Graph</a></td></h6></pre></tr>"
}
# send to csv file everything...
If ($PingStatus.StatusCode -eq 0)
{Add-Content "c:\scripts\RESULTS\$MachineName.csv" ($MachineName + ",up,"+ $RunDate +","+ $PingTime+","+ $status +","+$timeout+","+$ttl+","+ $RTTL +","+$ri+","+$RT)}
Else
{Add-Content "c:\scripts\RESULTS\$MachineName.csv" ($MachineName + ",down," + $RunDate +","+ $PingTime+","+ $status +","+$timeout+","+$ttl+","+ $RTTL +","+$ri+","+$RT)}
# $MachineName to $csv to put all results in one file
}
#put all Up results at end of file
$data = (get-content $up)
$htmlContent += $data + "</table>"
#Need to close the syntax of the HTML properly
$htmlContent += "</body></html>"
clear-content -Path $pingResults
$htmlContent | Out-File $pingResults

 

What’s Next?

The next phase of this project is to get away from the CSV files, as they are going to get extremely bloated.  We want to run the ping.ps1 script about every 15 seconds, which will generate 4 times the amount of data.  So what I want to do is break out the ping.ps1 script into two separate scripts.  The first will be used to perform the actual pings, writing the results to a database, and then the second will read the results from the database to generate the HTML table of up/down results; which will also before formatted differently and contain some additional information.  The GeneratePingCharts.ps1 script will then be modified to obtain the results from the database instead of the CSV file.

Portfolio & My Story

I have made some major overhauls to my website over the past month to a point where I can deem things acceptable.  The latest updates include finding a portfolio implementation I like and getting an initial start on my story, where I have a start on the academics section but am still trying to figure out what to write and how to word parts of it.  I have a solid section for my athletics to give a comprehensive overview.  I’m not really sure if I need to give any more details like injuries, additional mini-stories, etc.

The plugin I came across which give me the best look and feel for what I am going for is the Aeolus Portfolio WP Plugin.  This wasn’t as simple as install and configure.  The latest version, as of this writing, is 1.8 which once installed and activated did not work properly with my theme.  The issue I was having is that all the content would become ‘minified’ and illegible due to the plugin altering the display of the theme.  My best guess is that this is due to the Bootstrap update for the plugin.

In order to get around this I removed the plugin I installed through the WordPress Add Plugin feature and grabbed version 1.7 of the Aeolus Creative Portfolio plugin, unzipped the file and uploaded the folder to my WordPress installation plugins directory.  I was then able to go to the Plugins section on the dashboard within WordPress and activate version 1.7 of the plugin.  This did not cause any negative effects with my theme so I continued on and configured my Portfolio page to display all portfolio items with pagination similar to a parallax effect since it gives the short description for items.  I’m not sure at the moment how well this is going to work long term as it doesn’t appear to show categories; however, you can filter by category so I may be able to do one of the following once there are a lot of items:

  • Add multiple portfolio short codes to one page filtered by category and then add a top navigation manually  with anchor tags / headings to the various sections
  • Add individual pages for each portfolio category as subpages of Portfolio and the root page would be either like the first bullet point or links with explanations to the sub pages

Now I just have to build up my portfolio.  This all started with a portfolio being a requirement for a job interview I had while finishing up my Masters.  This is why the only portfolio items so far are for my academic projects.  Presently I have no work projects I am allowed to divulge (or for that matter have access to anymore) from past or present positions.

In the case of personal projects the portfolio items will appear upon completion of the first major version (1.0) and will be updated with any major features that get added as well as link to the corresponding category containing all posts pertaining to the project.

Transition to Quark Theme

I have been working with the Revera theme for about 4 months now I have discovered that the theme does not work for me as well as I would have liked.  This is mostly due to the main page slider and needing a reasonably sized images, where ideally they are all the same to prevent annoying page movements, and since I’m not running a photo based blog the theme does not make sense for me.

I have been casually looking around for a replacement theme that contained a clean look, is responsive, and customizable.  I came across Quark and like how the theme is built upon HTML5 and CSS3 and according to some of the reviews the code isn’t bloated and the theme is easy to customize, which I am going to venture into over the next week.

I have already started some basic, but key, code level modifications by adding a modification of the default full page template to include one that does not allow for comments as I do not want to allow users to comment on pages.   The one thing this theme doesn’t have that I need is a portfolio section, which leads me into the key elements I need to complete to get this theme working for me:

  • Portfolio template to allow for a section to showcase my applications
  • Full page template without comments (done)
  • Front page and footer widget areas the theme allows you to set
  • Front page display in general
  • Logo for my website
    • Main logo with website name and potentially caption
    • Favicon

I think this theme has the potential to work nicely for me so long as a theme update does not break my code level customizations which at this point appears to be the addition of two new page templates.  I am going to do further reading on the theme to learn more about what Quark has to offer.  The theme does not allow you to bookmark posts based on the short links (with the ?p=#) so I may need to find a way to add that too, but just encase I made the modification now to change the permalink structure for blog posts to /blog/%postname%/, which unfortunately will break any current links that are not short links.

Separation of Concerns: View Model & Database Model

I noticed how when I used scaffolding to generate my MVC5 controller with views using Entity Framework this would only allow me to specify a model representing a table in my database, which is not what I want.  In order to have true separation of concerns I wanted to:

  • Use view models for my views that get validated prior to any database updates
  • Use database models, auto generated, to represent my database and contain the results of queries

I accomplished this by implementing various services which handle the communication with the database.  The controller contains an instance of the service it requires to communicate with the database.  The service queries the database and returns the results to the controller, which then sends the information to the view utilizing the appropriate format.

When querying (performing  select) the database:

  • Resulting database model(s) are converted to the corresponding view model(s), and
  • Returned to the controller, where
  • They are passed through to the view for display

When updating an entry:

  • View Model is passed from the view to the controller
  • Validated using an appropriate validator, I built with Fluent Validation
  • If invalid, error message is returned to the view and displayed
  • If valid, view model is passed to the service to update the corresponding tables in the database
  • In either case if errors occur then the model state is updated and the default view for the action is returned

An example of this is how I manage reading lists.  I have my reading list controller (ReadingListController class) which handles user interactions and manipulates the model.  You can see below that the controller class contains minimal logic and utilizes helper classes to perform the logic. A validator class is used for validating data being passed from the view, when creating/editing a reading list, using Fluent Validation.  This ensures valid data is being entered into the database rather than relying on exceptions being returned on insert/updating of the database due to ill formatted values.

To provide separation of concerns services are used to interact with the database model.  Utilizing a service also means that the controller does not need to know the implementation of the service and how to interact with the database so by extending the particular service interface you can change the implementation, i.e. if I decided to not utilize the Entity Framework anymore and instead use NHibernate.

In the case of reading lists two services are used, the reading list service and the author service. The author service is utilized here so that we do not duplicate code for retrieving authors who have books and specific books for that author based on selection.  I am still working on the view implementation of this.  The reading list service contains the implementation for retrieving:

  • List of reading lists for a particular user (List title, number of novels in the list)
  • A specific reading list (for editing/deleting)
  • Book information for a book in a particular reading list
  • Details for a reading list (Book titles, year, whether they are read, and date finished)

Also an implementation for editing, creating, and deleting a reading list.

[Authorize]
public class ReadingListController : Controller
{
	private readonly IReadingListService _service;
	private readonly IAuthorService _aService;
 
	public ReadingListController() : this(new ReadingListService()) { }
 
	public ReadingListController(IReadingListService service)
	{
		_service = service;
	}
 
	// GET: ReadingList
	public ActionResult Index()
	{
		int userId = GetUserId();
 
		var readingLists = _service.GetReadingListsForUser(userId);
 
		if(readingLists == null)
		{
			ModelState.AddModelError("", "An error occurred in retrieving your reading lists.");
			return View();
		}
 
		return View(readingLists);
	}
 
	// GET: ReadingList/Details/5
	public ActionResult Details(int? id)
	{
		if (id == null)
		{
			return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
		}
 
		var readingListDetails = _service.GetReadingListDetails(id ?? 0);
 
		if (readingListDetails == null)
		{
			return HttpNotFound();
		}
 
		return View(readingListDetails);
	}
 
	// GET: ReadingList/Create
	public ActionResult Create()
	{
		return View();
	}
 
	// POST: ReadingList/Create
	// To protect from overposting attacks, please enable the specific properties you want to bind to, for 
	// more details see http://go.microsoft.com/fwlink/?LinkId=317598.
	[HttpPost]
	[ValidateAntiForgeryToken]
	public ActionResult Create([Bind(Include = "ListTitle")] ReadingListModel readingListModel)
	{
		readingListModel.UserId = GetUserId();
		var rlValidator = new ReadingListValidator();
		var results = rlValidator.Validate(readingListModel);
 
		if (results.IsValid)
		{
			var success = _service.CreateReadingList(readingListModel);
 
			if (!success)
				return RedirectToAction("Failed", new FailedModel { Message = "Unable to create reading list, please try again.", Action = "Index" });
 
			return RedirectToAction("Index");
		}
 
		return View(readingListModel);
	}
 
	public ActionResult Failed(FailedModel fm)
	{
		return View(fm);
	}
 
	// GET: ReadingList/Edit/5
	public ActionResult Edit(int? id)
	{
		if (id == null)
		{
			return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
		}
 
		var readingList = _service.GetReadingList((int)id);
 
		if(readingList != null)
			readingList.UserId = GetUserId();
 
		//ViewBag.UserID = new SelectList(db.Users, "UserID", "Username", readingList.UserID);
		return View(readingList);
	}
 
	// POST: ReadingList/Edit/5
	// To protect from overposting attacks, please enable the specific properties you want to bind to, for 
	// more details see http://go.microsoft.com/fwlink/?LinkId=317598.
	[HttpPost]
	[ValidateAntiForgeryToken]
	public ActionResult Edit([Bind(Include = "ReadingListID,ListTitle,UserID")] ReadingListModel readingListModel)
	{
		var rlValidator = new ReadingListValidator();
		var results = rlValidator.Validate(readingListModel);
 
		if (results.IsValid)
		{
			var success = _service.EditReadingList(readingListModel);
 
			if (!success)
				return RedirectToAction("Failed", new FailedModel { Message = "Unable to edit reading list, please try again.", Action = "Index" });
 
			return RedirectToAction("Index");
		}
 
		return View(readingListModel);
	}
 
	// GET: ReadingList/Delete/5
	public ActionResult Delete(int? id)
	{
		if (id == null)
		{
			return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
		}
 
		var rlm = _service.GetReadingList((int)id);
 
		if (rlm == null)
		{
			return HttpNotFound();
		}
 
		return View(rlm);
	}
 
	// POST: ReadingList/Delete/5
	[HttpPost, ActionName("Delete")]
	[ValidateAntiForgeryToken]
	public ActionResult DeleteConfirmed(int id)
	{
		_service.DeleteReadingList(id);
		return RedirectToAction("Index");
	}
 
	//Get: ReadingList/DeleteBook/5/4
	public ActionResult DeleteBook(int? readingListId, int? bookId, string bookTitle)
	{
		if (readingListId == null || bookId == null)
		{
			return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
		}
 
		var bm = _service.GetReadingListBookInfo((int)bookId, (int)readingListId);
 
		if (bm == null)
		{
			return HttpNotFound();
		}
 
		return View(bm);
	}
 
	// POST: ReadingList/DeleteBook/5
	[HttpPost, ActionName("DeleteBook")]
	[ValidateAntiForgeryToken]
	public ActionResult DeleteBookConfirmed(int readingListId, int bookId)
	{
		_service.DeleteBookFromList(bookId, readingListId);
		return RedirectToAction("Details", new { id = readingListId });
	}
 
	// Get: ReadingLis/AddBook/5
	public ActionResult AddBook(int? readingListId, string rlTitle)
	{
		if(readingListId == null)
		{
			return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
		}
 
		var rlam = new ReadingListAuthorsModel
		{
			Authors = _aService.GetAuthorsWithBooks().ToList(),
			ListTitle = rlTitle,
			ReadingListId = (int)readingListId
		};
 
		return View(rlam);
	}
 
	public ActionResult Books(int authorId)
	{
		var books = _aService.GetBooksForAuthor(authorId);
 
		return Json(books, JsonRequestBehavior.AllowGet);
	}
 
	//Get: ReadingList/MoveBook/5/4
	public ActionResult MoveBook(int? readingListId, int? bookId, string bookTitle)
	{
		if (readingListId == null || bookId == null)
		{
			return new HttpStatusCodeResult(HttpStatusCode.BadRequest);
		}
 
		var bm = _service.GetReadingListBookInfo((int)bookId, (int)readingListId);
		var usersReadingLists = _service.GetReadingListsForUser(GetUserId()).Where(x => x.ReadingListId != readingListId);  //Can't move book to itself
 
		if (bm == null || usersReadingLists == null)
		{
			return HttpNotFound();
		}
 
		bm.ReadingLists.ToList().AddRange(usersReadingLists);
 
		return View(bm);
	}
 
	private int GetUserId()
	{
		int userId;
		userId = int.TryParse(User.Identity.Name.Split(':')[0], out userId) ? userId : 0;
		return userId;
	}
}

Validation for creating a new reading list is shown below.  I chose this example since it is a very simplistic validator, to implement.  The only criteria is that the list title has to be 1-255 characters long and the returned result for the number of novels in a list should be greater than or equal to zero, the highlighted rows.

using FluentValidation;
using tracker.Models.View.ReadingList;
 
namespace tracker.Validators.Novels
{
    public class ReadingListValidator : AbstractValidator<ReadingListModel>
    {
        public ReadingListValidator()
        {
            RuleFor(a => a.ListTitle).Length(1, 255).WithMessage("List name needs to be between 1 and 255 characters.");            RuleFor(a => a.NumberOfNovels).GreaterThanOrEqualTo(0).WithMessage("Number of novels was less than 0, please contact admin.");        }
    }
}

In order to utilize my view model instead of the database model for each view I changed the @model parameter to be my view model instead of the database model.   This ensures that the model is populated within the database session prior to being passed back to the view.  An example is the details view for a reading list:

@model tracker.Models.View.ReadingList.ReadingListDetailsModel

Upgrade to WordPress 4.0 Failed through GoDaddy

WordPress & Godaddy

I got myself into an interesting predicament today.  I was in going to show my mom how to update the church website and saw that there was an update to WordPress 4.0 showing in the GoDaddy console.  I decided to do the update and then that is when things went sideways.

Upon the update finishing I went to the website site only to see a 500 Internal server error message, shown below:

I knew this was not a good sign, and preceded to investigate and my first thought after some Googling was to manually upload WordPress 4.0 files (all of them except the wp-content folder & wp-config file), so I downloaded the zip and preceded to do so.  Once this didn’t fix my issue and the site is for someone else I wanted to rectify the problem as quickly as I could so I contacted GoDaddy support.  Thanks to the patience of Spencer D. and his technical contact we were able to get to the bottom of things.

The first step was to create a web.config file (back up any current one, or in my case where you deleted it, create a new one) that contains the following:

<?xml version="1.0"
encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <httpErrors errorMode="Detailed"
existingResponse="PassThrough"
/>
    </system.webServer>
</configuration>

In my case I am using the IIS 8 Plesk version; however, you should be able to find the correct version for you in this GoDaddy’s Help article if you are using GoDaddy Windows hosting.

You upload this to the root folder for your particular domain and in my specific case this led to me receiving the following when I navigated to the website, stlukesdryden.com, after waiting for a bit:

All-in-One Event Calendar: require(G:\PleskVhosts\dwcryan.com\stlukesdryden.com\wp-content\plugins\all-in-one-event-calendar\vendor\lessphp\lessc.inc.php): failed to open stream: No such file or directory @ G:\PleskVhosts\dwcryan.com\stlukesdryden.com\wp-content\plugins\all-in-one-event-calendar\lib\bootstrap\loader.php:88 #2
All-in-One Event Calendar: require(G:\PleskVhosts\dwcryan.com\stlukesdryden.com\wp-content\plugins\all-in-one-event-calendar\vendor\lessphp\lessc.inc.php): failed to open stream: No such file or directory @ G:\PleskVhosts\dwcryan.com\stlukesdryden.com\wp-content\plugins\all-in-one-event-calendar\lib\bootstrap\loader.php:88 #2
PHP Fatal error: require(): Failed opening required ‘G:\PleskVhosts\dwcryan.com\stlukesdryden.com\wp-content\plugins\all-in-one-event-calendar\vendor\lessphp\lessc.inc.php’ (include_path=’.;.\includes;.\pear’) in G:\PleskVhosts\dwcryan.com\stlukesdryden.com\wp-content\plugins\all-in-one-event-calendar\lib\bootstrap\loader.php on line 88

I now knew that an error in the All-in-One Event Calendar was causing my site to not be displayed.  In order to disable the failing plugin you can apparently either rename the plugin folder located in ../wp-content/plugins or (the way I did it) you can access your database through PhpMyAdmin and search the wp_options table for active_plugins.  I then saw this for the active_plugins option:

32       active_plugins       a:4:{i:0;s:55:”all-in-one-event-calendar/all-in-on…       yes

Editing this row, and focusing on the options_value column you want to delete the plugin information for the message you saw when navigating to your site earlier in this post, which in my case is All-in-One Event Calendar, in bold below:

a:4:{i:0;s:55:”all-in-one-event-calendar/all-in-one-event-calendar.php”;i:1;s:36:”contact-form-7/wp-contact-form-7.php”;i:2;s:24:”wordpress-seo/wp-seo.php”;i:3;s:31:”wp-google-maps/wpGoogleMaps.php”;}

It is important to delete the entire contents from the i to the ; (semicolon) for the plugin in question.  Once I had done this I was able to access my website homepage, though not the login screen (which I rectified by copying over the wp-login.php file, it wasn’t allowing me to copy earlier, because as I later realized I had the page, giving me the error, open in my browser), but no other pages on the site.  I was able to solve the broken links/pages in the same fashion as when I broke them with my publish, you can find it at the bottom of this post.

Worth noting is that Spencer mentioned my WordPress installation was indeed at version 4.0, which jives with what the GoDaddy console under ‘Manage My Web Applications’ states and when I access the WordPress Dashboard for my site it states I am running WordPress 4.0 too, so I’m hoping all is good now.  Well, that is except the Event Page is blank as I have to get the event calendar plugin working, which involves getting the latest version installed.

This process took longer than expected and I felt it was worth blogging about.  Will make sure I have a fair bit of free time on my hands before attempting to upgrade WordPress on this domain.

MySql.Data Reference

I previously wrote a post about configuring MySQL for use with a Microsoft MVC application.  I noticed after publishing that my application could not connect to the database and returned the following exception:

System.ArgumentException: The ADO.NET provider with invariant name ‘MySql.Data.MySqlClient’ is either not registered in the machine or application config file, or could not be loaded. See the inner exception for details. —> System.ArgumentException: Unable to find the requested .Net Framework Data Provider. It may not be installed.

In order to solve this I had to do two things:

  1. In the Solution Explore under ‘References’ set Copy Local to true for the MySql.data reference
  2. Add the database provider configuration to your Web.config as seen below, with your particular version
<system.data>
  <DbProviderFactories>
    <add name="MySQL Data Provider" 
         invariant="MySql.Data.MySqlClient" 
         description=".Net Framework Data Provider for MySQL"  
         type="MySql.Data.MySqlClient.MySqlClientFactory, MySql.Data, Version=6.4.4.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d" />
  </DbProviderFactories>
</system.data>

The reason for #2 is that I am trying to run my code on a machine where the provider is not installed, so I have to specify the provider in my configuration file and then installation adds it to the machine.config registering the provider on the remote server.

Configure MySQL with ADO.NET EF in VS 2013

MySQL, ADO.NET & MVC

Even though we are using a Microsoft framework (MVC) and ADO.NET Entity Framework (EF) we may not want to connect to a Microsoft Database.  In my case I am going to connect to a MySQL database which means I have to configure MySQL for use in my project.

I had to download the latest version of MySQL for Visual Studio installer and once downloaded run the installer and follow the on-screen prompts to install.  I went with the Typical install option.

I then downloaded the latest MySQL connector and ran the installer also using the Typical install option.

Once this is done you can open your project in Visual Studio and right click on the solution in the solution explorer to Manage NuGet Packages… as in Figure 1 below.

Figure 1: Manage NuGet Packages
Figure 1: Manage NuGet Packages

You then go to the Online section and search for MySQL.  You then want to install the MySQL.Data.Entities package, highlighted in Figure 2 below, by clicking install which will then install the NuGet package and it’s dependencies (Figure 3) and once it’s done a green check mark will show that the package installed correctly (Figure 4).

Figure 2: Search MySQL and Select
Figure 2: Search MySQL and Select
Figure 3: Installing MySQL and Dependencies
Figure 3: Installing MySQL and Dependencies
Figure 4: Shows Installed - Close NuGet
Figure 4: Shows Installed – Close NuGet

After you have MySQL configured you need to add the MySQL provider to your Web.config by replacing the Entity Framework tag with the following:

<entityFramework>
     <defaultConnectionFactory type="MySql.Data.Entity.MySqlConnectionFactory, MySql.Data.Entity.EF6"/>
     <providers>
        <provider invariantName="MySql.Data.MySqlClient" type="MySql.Data.MySqlClient.MySqlProviderServices, MySql.Data.Entity.EF6"/>
     </providers>
</entityFramework>

Then you save the Web.config file and add an ADO.NET Entity Model by adding a new item (Figure 5) to your solution.  The new item is an ADO.NET Entity Model which provides the connection to your database (Figure 6) by selecting Visual C# from the left menu followed by ADO.NET Entity Data Model, giving it a name (in my case TrackerDB) and clicking Add.

Figure 5: Add New Item to Solution
Figure 5: Add New Item to Solution
Figure 6: Select Visual C# ADO.NET Entity Data Model
Figure 6: Select Visual C# ADO.NET Entity Data Model

In the window that appears select the option EF Designer from Database and click Next (Figure 7).  In the window that appears, Figure 8, click the New Connection button to create a connection to your database and select MySQL Database from the list of options, Figure 9.  Make sure to deselect the option to always use this selection and click Continue.

 

Figure 7: Select EF Designer From Database
Figure 7: Select EF Designer From Database
Figure 8: New Connection
Figure 8: New Connection
Figure 9: Select MySQL Database & Uncheck Always use this selection
Figure 9: Select MySQL Database & deselect ‘Always use this selection’

The Connections Properties menu will open where you will need to provide the details for the database server your application is going to connect to, as in Figure 10.  In my case I entered the server name as an ip address since no DNS friendly name exists along with the username and password required to connect to the database server.  You can click the Test Connection button to ensure your information is correct; however, in my case I also noticed that you can attempt to select the database you want to.  If your credentials are incorrect an error message will appear, otherwise a list of databases the username/password combination has access to will appear.    If the connection succeeds click OK, returning you to the previous screen.

Figure 10: Enter MySQL Database Information
Figure 10: Enter MySQL Database Information

As an aside I find it best to create a username/password combination that is unique to each database, rather than one to rule them all, so that if the credentials get compromised only access to a particular database will be granted.  If you have finer grained control over database usernames it is best to only grant a user the rights to do the minimal tasks required.

After clicking OK I received an error message (Figure 11) about the XML declaration.  I clicked OK which brings you back to a screen like Figure 8 above, however, information will be populated under the connection string.  I determined this was caused by me having extra lines, which I had commented out as part of my experimentation, at the top of my Web.Config file.  After deleting these lines making the <?xml …> tag the first line in the file I was able to proceed (canceled, deleted the connection and started from the beginning) without error which brought me back to the screen as seen in Figure 12.  I stuck with the default name provided for the connection string in my Web.Config file and choose yes for storing my password since the account only has the writes required.  I may decide to change this later and how you wish to handle this is up to you.  You then want to click the Next button.

Figure 11: XML Error Message - Click OK
Figure 11: XML Error Message – Click OK
Figure 12: Yes/No to Password and Connection Setting
Figure 12: Yes/No to Password and Connection Setting

Update (Aug 22, 2014): You may want to provide a better name for your connection settings in your Web.Config file as that acts as your context.  That is if your database is called tracker then db_nameEntities could be TrackerContext.

On the next screen you will see “Retrieving database information, please wait..” followed by the various elements you can select to be included in your model.  In my case I selected only Tables as I currently have no Views or Stored Procedures and Functions for my database, as seen in Figure 13.  I left the default to pluralize or singularize generated object names as well as to include foreign keys since I want my model to include the foreign keys.  Click Finish to complete the Entity Data Model Wizard.

Update (Aug 22, 2014): You may want to provide a more friendly model name for your database model, again if database is called Tracker you could use TrackerModel.

Figure 13: Select Database Object and Finish
Figure 13: Select Database Object and Finish

You will receive the message “Running this text template can potentially harm your computer. Do not run if you obtained it from untrusted source.. ”, as a reminder that you are about to execute someone else’s code which you should only do if you trust the source of that code.  I am confident in this case that I can run this template so I click OK, which I had to do a couple of times.  If you want you can disable these warnings by going to Tools Options… Text Templating and setting the Show Security Message to False.

Figure 14: Click OK to Run Text Template
Figure 14: Click OK to Run Text Template

Under Data Connections in the Solution Explorer I able to see the database I added.  At first it showed as having a red ‘X’ next to the database icon, but once I expanded the connection to see the contents, Figure 15, the icon refreshed and the database contents were displayed which signifies that I now have a data connection established to my database for use with my application.

Figure 15: View Database Contents to Confirm
Figure 15: View Database Contents to Confirm

Finally now that everything is configured you are able to see an ERD diagram of you database open up as well as the database schema in the Solution Explorer that you can navigate, shown in Figure 16.

Figure 16: Tracker DB in Solution Explorer and Diagram
Figure 16: Tracker DB in Solution Explorer and Diagram

The connection string added to my Web.Config file under the <connectionStrings> tag looks similar to below, but with the actual values for server, user id, and database.  The reason for the connection string (with the space) and the &quot sections is because it is escaping the connection string generated for the database I am connecting to and putting it into the web configuration file under the connectionString attribute.  I will have to see if this works as generated by the wizard or if I will have to modify its results.

<add name="tracker_appEntities" connectionString="metadata=res://*/TrackerDB.csdl|res://*/TrackerDB.ssdl|res://*/TrackerDB.msl;provider=MySql.Data.MySqlClient;provider connection string=&quot;server=192.168.2.1;user id=user_name;password=db_password;persistsecurityinfo=True;database=db_name&quot;" providerName="System.Data.EntityClient" />

Note:  You may not run into this, however, there were times throughout when Visual Studio would go into a non-responsive state (sometimes flashing in and out of them) throughout the process.  I found that I just had to be patient and give the application time to complete the necessary tasks.

Publish Web Project to GoDaddy

The web hosting provider I am using is GoDaddy and I chose Windows hosting since some of my ideas include developing web applications using Microsoft technologies.  The following explains how I set up my first web project I am in the process of developing, which was my tracker application.

The first step is to get the web deploy publishing settings for the location you wish to host your web project. In my case I want to host it on a subdomain (which I have already created), but the steps are the same if it is your root domain. You navigate to the subdomain section of where you would like to host your app in GoDaddy’s ‘Websites & Domains’ section.  You then click the ‘Web Deploy Publishing Settings’ link to start the download of your settings file (seen in Figure 1 below).

Step 1 - Download Web DeployPublishing Settings
Figure 1: Download Web Deploy Publishing Settings

The downloaded file will contain a .publishsettings extension with the contents of the file similar to:

<?xml version="1.0"?>
<publishData><publishProfile profileName="tracker.dwcryan.com - Web Deploy" publishMethod="MSDeploy" publishUrl="tracker.dwcryan.com" msdeploySite="tracker.dwcryan.com" userName="dwcryan" destinationAppUrl="http://tracker.dwcryan.com" controlPanelLink="https://p3nwvpweb100.shr.prod.phx3.secureserver.net:8443"/></publishData>

Once you have the settings file you can configure your actual project. With your web project opened in Visual Studio you right click on the project in the Solution Explorer and select Publish.. (or something similar depending on project type) as in Figure 2.

Step 2 Right Click Project In Solution Explorer and select Publish
Figure 2: Right Click Project In Solution Explorer and select Publish

This will present you with the window in Figure 3, where you will select the Import option by left clicking.  In the displayed dialog ‘Import Publish Settings’, also displayed in Figure 3, you will browse to your publish settings file, select it, and then click Open; thus, displaying the path to your file in the ‘Import Publish Settings’ dialog as in Figure 4, and then click OK.

Step 3 - Click Import
Figure 3 – Click Import
Step 4 - Browse For Publishing File
Figure 4: Browse For Publishing File

Once the settings have loaded enter the password for your account and click Test Connection.  If everything is OK you will see a check mark next to the test connection button as in Figure 5.

Step 5 - Enter Password and Validate Connection
Figure 5: Enter Password and Validate Connection

Clicking Next will then take you to the ‘Settings’ tab.  I did not change anything here as I did not see the need for any file publishing options and I do not want the publish to modify my database.  I may need to update my publish settings and insert a connection string for my database once I have that configured within my application and will update this post with any such changes.  Clicking Next again brings me to the ‘Preview’ tab as in Figure 6.

Step 6 - Preview and Publish
Figure 6: Preview and Publish

Clicking ‘Start Preview’  in the middle of the screenshot above (Figure 6) will provide a list of the files being uploaded and the action being performed (i.e. update for files being modified and add for new files being pushed to the destination) as in Figure 7.

Step 7 - Clicking Preview
Figure 7: Clicking Preview

Lastly you click Publish to finish off the process which will publish your application to its destination.  The output of the file will look similar to that of Figure 8 (I cropped out the middle section as to highlight the start and end sections as well as displaying some of the adds and updates.  The web application (at the URL provided) will then open up in your default web browser.

Step 8 - Output
Figure 8: Output

If their are issues with the settings in your application when running on the remote server an error page will be displayed with some information as to what you have to do.  At this point it becomes a matter of experimenting.  In my case I got a security exception stating “The application attempted to perform an operation not allowed by the security policy. To grant this application the required permission please contact your system administrator or change the application’s trust level in the configuration file”.

I was able to solve this by adding the following line to my web.config file between the system.web tags and publishing again:

<trust level="Full" />

Through further investigative work I determined a second option is changing the configuration on your hosting provider, if they allow it. In my case that involved selecting the option ASP.NET Settings in the same view as Figure 1 above, which in my case was at the bottom of the 3rd column (from the left).  Once in there you set the CAS trust level to Full, Figure 8 below, and while here make sure that the Default webpage language is correct for your application.

CAS trust level
Figure 8: CAS trust level

That appears to be it for now and will have to see what happens once my web application contains a database and actions are being taken if any further modifications to the publishing settings need to be added or modified.