I previously wrote a post about configuring MySQL for use with a Microsoft MVC application. I noticed after publishing that my application could not connect to the database and returned the following exception:
System.ArgumentException: The ADO.NET provider with invariant name ‘MySql.Data.MySqlClient’ is either not registered in the machine or application config file, or could not be loaded. See the inner exception for details. —> System.ArgumentException: Unable to find the requested .Net Framework Data Provider. It may not be installed.
In order to solve this I had to do two things:
In the Solution Explore under ‘References’ set Copy Local to true for the MySql.data reference
Add the database provider configuration to your Web.config as seen below, with your particular version
<system.data><DbProviderFactories><addname="MySQL Data Provider"invariant="MySql.Data.MySqlClient"description=".Net Framework Data Provider for MySQL"type="MySql.Data.MySqlClient.MySqlClientFactory, MySql.Data, Version=184.108.40.206, Culture=neutral, PublicKeyToken=c5687fc88969c44d"/></DbProviderFactories></system.data>
<add name="MySQL Data Provider"
description=".Net Framework Data Provider for MySQL"
type="MySql.Data.MySqlClient.MySqlClientFactory, MySql.Data, Version=220.127.116.11, Culture=neutral, PublicKeyToken=c5687fc88969c44d" />
The reason for #2 is that I am trying to run my code on a machine where the provider is not installed, so I have to specify the provider in my configuration file and then installation adds it to the machine.config registering the provider on the remote server.
Even though we are using a Microsoft framework (MVC) and ADO.NET Entity Framework (EF) we may not want to connect to a Microsoft Database. In my case I am going to connect to a MySQL database which means I have to configure MySQL for use in my project.
I had to download the latest version of MySQL for Visual Studio installer and once downloaded run the installer and follow the on-screen prompts to install. I went with the Typical install option.
I then downloaded the latest MySQL connector and ran the installer also using the Typical install option.
Once this is done you can open your project in Visual Studio and right click on the solution in the solution explorer to Manage NuGet Packages… as in Figure 1 below.
You then go to the Online section and search for MySQL. You then want to install the MySQL.Data.Entities package, highlighted in Figure 2 below, by clicking install which will then install the NuGet package and it’s dependencies (Figure 3) and once it’s done a green check mark will show that the package installed correctly (Figure 4).
After you have MySQL configured you need to add the MySQL provider to your Web.config by replacing the Entity Framework tag with the following:
Then you save the Web.config file and add an ADO.NET Entity Model by adding a new item (Figure 5) to your solution. The new item is an ADO.NET Entity Model which provides the connection to your database (Figure 6) by selecting Visual C# from the left menu followed by ADO.NET Entity Data Model, giving it a name (in my case TrackerDB) and clicking Add.
In the window that appears select the option EF Designer from Database and click Next (Figure 7). In the window that appears, Figure 8, click the New Connection button to create a connection to your database and select MySQL Database from the list of options, Figure 9. Make sure to deselect the option to always use this selection and click Continue.
The Connections Properties menu will open where you will need to provide the details for the database server your application is going to connect to, as in Figure 10. In my case I entered the server name as an ip address since no DNS friendly name exists along with the username and password required to connect to the database server. You can click the Test Connection button to ensure your information is correct; however, in my case I also noticed that you can attempt to select the database you want to. If your credentials are incorrect an error message will appear, otherwise a list of databases the username/password combination has access to will appear. If the connection succeeds click OK, returning you to the previous screen.
As an aside I find it best to create a username/password combination that is unique to each database, rather than one to rule them all, so that if the credentials get compromised only access to a particular database will be granted. If you have finer grained control over database usernames it is best to only grant a user the rights to do the minimal tasks required.
After clicking OK I received an error message (Figure 11) about the XML declaration. I clicked OK which brings you back to a screen like Figure 8 above, however, information will be populated under the connection string. I determined this was caused by me having extra lines, which I had commented out as part of my experimentation, at the top of my Web.Config file. After deleting these lines making the <?xml …> tag the first line in the file I was able to proceed (canceled, deleted the connection and started from the beginning) without error which brought me back to the screen as seen in Figure 12. I stuck with the default name provided for the connection string in my Web.Config file and choose yes for storing my password since the account only has the writes required. I may decide to change this later and how you wish to handle this is up to you. You then want to click the Next button.
Update (Aug 22, 2014): You may want to provide a better name for your connection settings in your Web.Config file as that acts as your context. That is if your database is called tracker then db_nameEntities could be TrackerContext.
On the next screen you will see “Retrieving database information, please wait..” followed by the various elements you can select to be included in your model. In my case I selected only Tables as I currently have no Views or Stored Procedures and Functions for my database, as seen in Figure 13. I left the default to pluralize or singularize generated object names as well as to include foreign keys since I want my model to include the foreign keys. Click Finish to complete the Entity Data Model Wizard.
Update (Aug 22, 2014): You may want to provide a more friendly model name for your database model, again if database is called Tracker you could use TrackerModel.
You will receive the message “Running this text template can potentially harm your computer. Do not run if you obtained it from untrusted source.. ”, as a reminder that you are about to execute someone else’s code which you should only do if you trust the source of that code. I am confident in this case that I can run this template so I click OK, which I had to do a couple of times. If you want you can disable these warnings by going to Tools > Options… > Text Templating and setting the Show Security Message to False.
Under Data Connections in the Solution Explorer I able to see the database I added. At first it showed as having a red ‘X’ next to the database icon, but once I expanded the connection to see the contents, Figure 15, the icon refreshed and the database contents were displayed which signifies that I now have a data connection established to my database for use with my application.
Finally now that everything is configured you are able to see an ERD diagram of you database open up as well as the database schema in the Solution Explorer that you can navigate, shown in Figure 16.
The connection string added to my Web.Config file under the <connectionStrings> tag looks similar to below, but with the actual values for server, user id, and database. The reason for the connection string (with the space) and the " sections is because it is escaping the connection string generated for the database I am connecting to and putting it into the web configuration file under the connectionString attribute. I will have to see if this works as generated by the wizard or if I will have to modify its results.
Note: You may not run into this, however, there were times throughout when Visual Studio would go into a non-responsive state (sometimes flashing in and out of them) throughout the process. I found that I just had to be patient and give the application time to complete the necessary tasks.
The web hosting provider I am using is GoDaddy and I chose Windows hosting since some of my ideas include developing web applications using Microsoft technologies. The following explains how I set up my first web project I am in the process of developing, which was my tracker application.
The first step is to get the web deploy publishing settings for the location you wish to host your web project. In my case I want to host it on a subdomain (which I have already created), but the steps are the same if it is your root domain. You navigate to the subdomain section of where you would like to host your app in GoDaddy’s ‘Websites & Domains’ section. You then click the ‘Web Deploy Publishing Settings’ link to start the download of your settings file (seen in Figure 1 below).
The downloaded file will contain a .publishsettings extension with the contents of the file similar to:
<?xmlversion="1.0"?><publishData><publishProfileprofileName="tracker.dwcryan.com - Web Deploy"publishMethod="MSDeploy"publishUrl="tracker.dwcryan.com"msdeploySite="tracker.dwcryan.com"userName="dwcryan"destinationAppUrl="http://tracker.dwcryan.com"controlPanelLink="https://p3nwvpweb100.shr.prod.phx3.secureserver.net:8443"/></publishData>
Once you have the settings file you can configure your actual project. With your web project opened in Visual Studio you right click on the project in the Solution Explorer and select Publish.. (or something similar depending on project type) as in Figure 2.
This will present you with the window in Figure 3, where you will select the Import option by left clicking. In the displayed dialog ‘Import Publish Settings’, also displayed in Figure 3, you will browse to your publish settings file, select it, and then click Open; thus, displaying the path to your file in the ‘Import Publish Settings’ dialog as in Figure 4, and then click OK.
Once the settings have loaded enter the password for your account and click Test Connection. If everything is OK you will see a check mark next to the test connection button as in Figure 5.
Clicking Next will then take you to the ‘Settings’ tab. I did not change anything here as I did not see the need for any file publishing options and I do not want the publish to modify my database. I may need to update my publish settings and insert a connection string for my database once I have that configured within my application and will update this post with any such changes. Clicking Next again brings me to the ‘Preview’ tab as in Figure 6.
Clicking ‘Start Preview’ in the middle of the screenshot above (Figure 6) will provide a list of the files being uploaded and the action being performed (i.e. update for files being modified and add for new files being pushed to the destination) as in Figure 7.
Lastly you click Publish to finish off the process which will publish your application to its destination. The output of the file will look similar to that of Figure 8 (I cropped out the middle section as to highlight the start and end sections as well as displaying some of the adds and updates. The web application (at the URL provided) will then open up in your default web browser.
If their are issues with the settings in your application when running on the remote server an error page will be displayed with some information as to what you have to do. At this point it becomes a matter of experimenting. In my case I got a security exception stating “The application attempted to perform an operation not allowed by the security policy. To grant this application the required permission please contact your system administrator or change the application’s trust level in the configuration file”.
I was able to solve this by adding the following line to my web.config file between the system.web tags and publishing again:
<trust level="Full" />
Through further investigative work I determined a second option is changing the configuration on your hosting provider, if they allow it. In my case that involved selecting the option ASP.NET Settings in the same view as Figure 1 above, which in my case was at the bottom of the 3rd column (from the left). Once in there you set the CAS trust level to Full, Figure 8 below, and while here make sure that the Default webpage language is correct for your application.
That appears to be it for now and will have to see what happens once my web application contains a database and actions are being taken if any further modifications to the publishing settings need to be added or modified.
Well, today I was doing some experimenting and investigating how I could get my Tracker application published to a subdomain through automation instead of manually copying the files every time. It’s only an MVC template right now but I wanted to be able to test that the default functionality was going to work with my web hosting, prior to proceeding. Things were going pretty good until I realized that publishing my application broke my WordPress site. By broke what I mean is that critical files and settings got overwritten causing my site navigation menu to get messed up as well as links to pages not working (404 errors) and only my home page could be displayed.
I hypothesized the cause of the overwrite to be due to the subdomain folder being contained within the root folder for my website instead of up one level so I deleted the subdomain and the folder it was using. I then recreated it so its root folder is at the same level as my main sites root folder. Apparently trying to organize your subdomains within your root domain folder is not a good idea.
I was then able to follow the steps again to publish a web application, which I outline here.
I took the opportunity of my site already being broken to install the latest WordPress update.
Since my incorrect publish broke my custom links I determined how to fix this was from my WordPress dashboard you go to Settings -> Permalinks and switch the option to Default and save your changes. Then you switch it back to your custom selection and save the changes again. Prior to discovering this not even deleting the pages permanently and recreating them removed the 404 error for me.
What is an application without some sort of storage mechanism behind the scenes? Above represents my database design which I am going to go into more details about.
At the heart of the database is the User table which stores user information and associates a reading list, measurement, or golf round with a corresponding user such that users can only see their data when logged in. This also makes it easy to track additional information by adding the corresponding tables which will then get associated with a user if I choose to expand the application at a later time. The other tables which stores, what I will call public data, is visible to all users to avoid duplication of data. This includes golf course information (Golf Course, Address, Region and Course Par Information tables) and book information (Author, Book, and what authors are associated with which book). The measurement and course par info tables each have a ‘special’ column. The measurement table contains a column called Type which takes a number from 1-4. Since this never changes the application can handle the numbers, rather than creating an additional static look-up table in the database. The number representation is as follows:
The course par information table contains a column called IsCNine to accommodate courses that have more than 18 holes with a format of: (Course Color)*( )(Front|Back). Where there can be 0 or more colors Examples of this would be:
This format allow for courses which provide 27 holes of golf as they typically have a white, blue, and red course, each with 9 wholes. I’d imagine both would also have a front and back nine. I have never played at a 27 whole golf course but I try to develop for future (or potential) possibilities.
I’ll explain the user profile dynamic content and dashboards text boxes in the diagram in my next post about the dynamic content and dashboards I plan to provide in my application.
Since not all my posts will contain a feature image I had to find a work around that would allow the text content to show properly in the slider. In order to do this I added a very tiny white square as my featured image. When it first displayed the caption for the post was at the bottom and ran out of the visual area (couldn’t even see ‘Read More’ button). In order to accommodate this I modified the CSS for the slider so that the caption starts at 20% from the top instead of its default of 40%; in hopes this will also work for future blog posts without a featured image.
I had to change a block of CSS which starts at line 235 in theme.css from:
It is worth noting that a caveat I have discovered is that this is insufficient for smaller screen sizes so I may have to become more ‘clever’ in my featured images; like I did for this post, when I can.
Ahh, it is so nice to have completed my Masters and be able to focus on some personal projects I have been pondering. The first was this website and the second is a tracker application. I have been using an Excel spreadsheet I store on Google Drive for tracking various things, which does work pretty well for just maintaining one long list of, for instance, the novels I have read and the novels I’d like to read. When it comes to tracking items with more complex items like rounds of golf played and at which course they were played dealing with the relationships becomes a little more cumbersome though I could also implement this in a spreadsheet; however, I thought this would be a great opportunity to learn more about web applications.
The technologies I will be using to develop my web app are C#.NET, Entity Framework, and MVC for the main framework of the application. I will also be utilizing CSS to style my application forms and views when necessary. I am going to attempt utilizing a MySQL back-end, since my web host provides more storage for MySQL databases, which requires additional configuration from utilizing MS SQL when using Microsoft technologies.
Security is an important aspect of any application making it necessary to implement a login system for my web app. The login form will be the first page a user sees when accessing the site. The web app will test, when access to other pages are attempted, if a valid user is logged in prior to granting access to any other page and redirect to the login page for unauthorized users.
Once the basic create, read, update, and delete (CRUD) functionality is implemented I am going to generate various dashboards for the users to provide statistics and further information at a glance based on database records relevant to the user. This will include elements like what was the last novel you read (and completion date) and the number of novels you read over a specified time frame. Related to golfing this would include calculating your overall golf handicap, average score for 9 and 18 whole games, and a further drill down to information per course will also be available. I’m thinking at least 5 rounds will have to be played for a handicap to be calculated for a given category and averages will always show as only courses you have played will be presented as options. I may also show the number of different courses you have played and a map of where they are located. Additional technologies I plan to use in accomplishing these tasks are Google Charts (Google Visualization API) and Google Maps API.
Ever since I decided to start up my own website (again) with the intent of having it for a long time I knew there were two key things I needed to do, besides the content itself. The first was to come up with my domain name and the second was to find a WordPress theme that I liked. I wanted the theme to contain some key features I was looking for as well to have a style that I liked. I spent time over the past month or so looking at themes and testing some I thought I might like before finally settling on Revera as a solid start to what I was looking for.
Revera is a free premium WordPress theme that contains many features that I was looking for. This includes being a responsive WordPress theme utilizing the Bootstrap 3 framework, custom menu, portfolio page template, and blog (which might not be so much a feature as ‘just is’). One feature it has that I am still uncertain as to how much I am going to like is featured images for your posts. I do like how they display prominently on the homepage to advertise new blog posts and display for portfolio elements; however, since I am not using it for photography there may be cases when an image in my post does not make sense, or that it should not be featured. I intend to utilize the portfolio section to showcase applications once a stable release is completed.
Since I discovered a theme that had a solid foundation for me to build off I progressed into performing some customizations to brand the theme for me and gain some additional functionality.
The first customization I performed was to ‘mini-brand’ my site by changing the colors to what I wanted, which was a darker shade of green as it is my favorite color. In order to do this I modified theme.css by changing the appropriate hex codes to the appropriate hex code for the shade of green I required. Below the left hand side represents the hex code for the default color and the right hand side represents my color choice. I also modified the color for the arrows on the homepage to match by modifying bg_direction_nav.png using MS Paint.
Above, inColor, is a special addition I made as I wanted to use links on the homepage feature section which has a background color the same as the default link color. In order to obtain this I added a child class for links called inColor:
I am not sure if I am going to stick with the red link color or not as I plan to get some opinions on the look of red on green. I at least have the CSS styles in place for if I want to change that color or have a different color for visited links.
Lastly I needed to add templates for full width pages, one with and one without a comment section. In both cases I also did not want the page title to duplicate. I accomplished this by modifying the page template, page.php, and creating two new template files full-page-no-comments.php and full-page-with-comments.php. The key here was stripping out the sections I did not need and in order to remove the title I had to modify the get_template_part function call to point to my new content template, which I called content-pageNoTitle.php. The naming is important as you can see the parallel between the parameters passed and the file name (left and right side of the hyphen). The function call becomes:
That’s it! Besides some basic text changes for various displays (i.e. blog section) that is all the code modifications and additions I made, to date, in order to get the theme to what I have envisioned for my virtual home.
This application imports electronic medical records (stored in text files) from the selected folder and runs each file through the natural language process of:
Removing stop words
Creating an index
For stemming the Porter Stemming Algorithm is implemented to transform a word into its root form and contains six steps:
Get rid of plurals and –ed or –ing endings
Turn terminal y to I when there is another vowel in the stem
Map double suffices to a single one
Deals with –ic-, -full, -ness etc. endings similarly to step 3
Takes off –ant, -ence, etc. in context of <c>vcvc<v>
Removes final –e if the measure of the number of constant sequences between 0 and j is greater than 1, i.e. <c>vcvc<v> gives 2
A trie (prefix tree) is an ordered data structure used to store the index in memory. All descendants of a node have a common prefix under the same branch. The common prefix eliminates the need for storing the same prefix each time which increases memory efficiency. A simple representation containing the words aspirin, ankle, and pain is:
When the user searches for keywords, Figure 1, spaces are seen as an AND condition and commas are seen as an OR condition. A list is displayed to the user showing the file names of the documents which contain there search terms. Double clicking on a particular document will open a new tab displaying the documents contents and highlighting the keywords present in the document, Figure 2 & 3. When mousing over the keywords found the SNOMED and UMLS IDs are displayed. These are determined based on a separate trie data structure built from the SNOMED CT data.
This application consists of three main parts which are the client, server, and the game of SOS. The server ran on an Elastic Cloud 2 (EC2) instance with Amazon Web Services where I configured the Windows server and supporting software. This includes creating a database in MS SQL Server, creating the necessary users and assigning them to the database they can access. Mappings are set for the user which grants the necessary permissions. The firewall also needed to be configured to allow the appropriate ports access so clients could connect to the server (and the server could receive connections). Internet Information Services (IIS) was also configured so that the Java Network Launch Protocol (JNLP) MIME type was recognized by the web server so that the client could be launched using Java WebStart from a webpage.
Client-Server communication uses a server socket that listens for incoming connections. When a connection is accepted a new thread (ClientHandler) is spawned which handles communications with the client through a socket. Java’s ObjectInputStream and ObjectOutputStream are used in order to send and receive Java objects.
Threading is used on the client side to listen for messages from the server. On the server side each client connection is ran in its own thread and handled by a ClientHandler class.
Database access is handled by the server using Java Database Connectivity (JDBC) drivers. Any information retrieved from the database is sent back to the client using a data transfer object (DTO) and packaged as a games room message of the correct type.
The application allows users to chat in a main room or create their own chat rooms which can be public or private. Users are able to join any public room but the only way a user can join a private room is through an invite from an active participant. Files can be sent to users of a room. This is done by reading the file as bytes and sending the byte array along with the filename as a string to users in the room. Java’s FileOutputStream class is used to write the file to the recipient’s local machine if they chose to accept the file and saved it.
Users are able to manage their profile information, Figure 1. Which allows them to change certain information, including updating their password. The application uses a hash implementation using the password-based key derivation function 2 (PBKDF2) algorithm with HMAC and SHA1 (part of RSA public-key cryptography standards) for hashing user passwords and then storing the hash value in the database. When a hash is being created for a password the string is converted to a byte array and a salt is created using java.security.SecureRandom for the specified salt byte size. The PBKDF2 hash of the password is then computed by creating a password based encryption (PBE) key of variable-key-size PBE ciphers utilizing the users’ password, salt, number of iterations, and how many bytes the result should be. Javas secret key factory, part of the javax cryptography package, is then utilized to generate the secret key (hash of the password) for the PBE key specification. All relevant data is then placed into one string which takes the format of ‘iterations:salt:hash’ where the salt and hash is first converted to hexadecimal. The colons are critical as they are utilized for splitting the information when validating passwords.
Users are able to play the game of SOS, Figure 2, when they are in a room they have created. The first two members to join the game can play, as it’s a two player game. The game implements the MVC pattern, using SosController.java as the controller, GameView.java as the view and the ChatClient as the model for how to handle requests from the user to the server and vise versa. DTOs are used to transfer data between the clients through the server as well as for saving game states. The DTOs are Board, Player, PlayerMove, SavedSosComponents, ScoreCard, and Turn.
Users are able to save the contents of a chat session for later viewing and a facility to view and manage any saved chats is provided, Figure 3. Users can also save a games state to be continued at a later date and when loading a saved games both participants must be logged into the application. Game states are converted to XML using XStream to be stored in the database.
Validators are created by implementing my Validate interface and a separate class for each validator (i.e. NewRoomValidator.java) is created. This separates out the process of validating information entered into the GUI from the actual GUI code itself.