Tag Archives: C#

Create a Word Cloud From Your Twitter Feed

I love playing with data. My data makes it even more fun. Wordle has been around for a long time, and so has Twitter (in Internet years anyways). I have always been fascinated by word clouds and visualizing text patterns, etc.

I figured that hey, there has got to be some analyzer for your twitter stream, and I am sure there are a ton, but I didn’t stumble upon any with some easy Googling, so I did it the hard way.

First goal? Get your Twitter feed and/or data somehow. Multiple ways to do this, but I stumbled upon a pretty cool site. http://tweetbook.in that let’s you create an eBook from your Twitter feed and favorites. It let’s you publish out as a PDF or XML file, so I figured that would work. It is a busy site and you may have to wait to get in but once you do you just oAuth it up to Twitter and grab your data.

Now, once you have your data, you need to do something with it. The data would be in XML so you need to parse out the data you want, for instance, I wanted to analyze my “favorites” so I wanted to get the text out of the XML. Here is my first favorite on Twitter (by the way, it will only go back 3200, I think – I only have 2600 or so faves)

  881539697
  A "Manager" class is like my grandmother's junk drawer.
  Fri Aug 08 14:23:56 +0000 2008
  web
  jeremydmiller
 

Well I just want to grab that <text> value, and without having to do any programming or powershell or C#, I fired up trusty old LINQPad (more on this tool in future posts for sure). I then just wrote a quick little query against the XML file like so:

var xml = XElement.Load (@"c:\fave.xml");

var query =
  from e in xml.Elements()
  select e.Element("text").ToString().Replace("","").Replace("","").Replace("RT ","");

query.Dump();

As you can see, I am just loading up the xml file and doing some text cleanup (removing the xml text blocks and removing RT’s, the old syntax which muddies up the results). Note in LINQPad you need to change the query type to C# Statements instead of the default C# Expression.

Once I had my values in the results I wanted, I did a quick CTRL+A, CTRL+C (it still baffles me how many people don’t know CTRL+A is “Select All”) and then pasted it into notepad++, to view, and cleaned up some html characters there (quotes, etc) and then pasted it into Wordle. Here is what I got back:

wordle_of_my_favorites

You can see I really like to favor SQL Server, Microsoft, iPhone, Blogs, sqlpass, Google, SharePoint, Twitter, and pretty much everything geeky. Pretty dang cool. Why doesn’t Twitter offer something like this? I think it would be cool. What other cool things have you done with your “data” – what cool things would you like to see?

Winforms DevExpress Grid

In the current agile sprint for the dev team I manage, we decided to start replacing ListViews (custom ones at that) and grids with the XtraGrid from DevExpress.

Now, normally I shy away from 3rd party controls, or want to vet them, but I knew there was no way we could do the same functionality in the default grid in .NET. One of the guys on the team did a story the sprint before to do a proof of concept comparing various grids and showing the pros and cons. DevExpress came out ahead, functionality and performance.

What we are seeing now is huge gains. We can use the new grid and functionality we couldn’t even begin to think of, is there by default. Grouping, searching, filtering, FAST performance, print preview and formatting, etc, etc, etc. Tons of options.

In fact, there are TOO many options. It makes it hard for us to digest all the possibilities in what we want to turn off, or how to integrate with existing forms, etc. A good problem to have.

All I know, is if you are doing any serious .NET work (Winforms, WPF, even Web/Silverlight) – it might make some sense to take a look at DevExpress. Focus on your business rules and integrating other parts of your systems, not reinventing the wheel with a crazy custom grid.

In the coming months I will try to talk about other areas that you can “outsource” to 3rd parties, where it makes sense. (And no, I don’t get anything from DevExpress for this post. Just calling it how I see it).

VB6 to .NET Migration: Preparation and Migration

In my previous post on this topic, VB6 to .NET Migration: Decision and Analysis, I talked about the decision to migrate instead of rewrite, and also the analysis portion of the project.

Once you are ready to go, what happens? You need to prepare for the migration process. What ArtInSoft needs in Test Cases and Test Scripts so they can run through your application on their end in QA (the VB6 version) and then once they migrate it, run through the same scripts again (once migrated to .NET).

My guess is shops have some kind of test scripts, but nothing too extensive. This is where we were scrambling, we had things all over the place and not very complete test scripts/cases. This ended up biting us in the end as a lot of the app wasn’t covered by test cases so it was hard for ArtInSoft to test and migrate it without tons of exploratory testing.

Since the migration, we have shored up our test scripts and test cases by using a QA firm, Beta Breakers, but that is for another post.

After ArtInSoft gets your test scripts, they run through them and ask any questions, and in the mean time, their dev team is migrating your code and looking for areas they can automated in the migration, that aren’t in the main migration scope of the their migration tool.

ArtInSoft sets up a SharePoint portal for your project so you can share documents and other things, as well as a discussion forum for “technical queries” – or things they want to know on a technical level what they should do during the migration.

As you can guess, many things in .NET aren’t the same as VB6, the way some controls react, etc. They want to know what to do in those cases. Also, you might have some crazy things going on in VB6 that have no direct translation to .NET, so they need to know what to do there, overall though it shouldn’t’ be too bad

Along the migration, you can get the migrated code as they are doing it every once in a while and look at it and get into source control on your end if you’d like. Also, up to to a certain point you can send minor changes to ArtInSoft, so you don’t have a to have a code freeze for the entire migration. Obviously at a given point, you can’t be sending changes, if you have them, you can do a “change control” request, but it is going to cost you, which is expected.

As they work on the migration, you can work on other things on your end, whatever you’d like. Towards the end of the migration (development), you will send 1-2 people down to Costa Rica to help finish the migration and deep dive into testing and development.

The final phase of the of the migration comes when your team becomes the “testing team” and you do User Acceptance Testing (UAT). You have a specified amount of time to test on your end and send bugs back to ArtInSoft. This is a good back and forth for 1-2 months possibly depending on the size of your application. Once you are OK with the code and app, you sign off and the code in your hands. Next up.. your own “fixing” period.

Japanese Character Sets – Input, Identification and Comparison

As a byproduct of some work our team is doing, we needed to test and handle different character sets for Asian languages (Katakana, Hiragana, etc). A guy on the team came up with a pretty sweet proof of concept test application so we can trial combinations and see how things work based on ignoring widths, etc. even some database interaction. We figured this would be useful to others in the .NET community so we are giving back

Check out http://jpcharsets.codeplex.com/ for more info, but basically you can play around with the different character sets to your hearts content and look at the source code behind the comparisons of different sets. Enjoy!

VB6 to .NET Migration: Decision and Analysis

Recently completed a huge project migrated a fairly decent size code base from Visual Basic 6 (VB6) to C# .NET. Yes, I said “migrated”, not rewrite – let’s get that out of the way up front. The code was migrated.

How did we go about doing this? Well, I’d like to detail it out over a few posts here and in the near future.

First, you need to make the decision to migrate instead of rewrite. Not very often do dev shops get the chance to do a rewrite, and even when they do, most rewrites go over time and budget and end up losing business logic and functionality, that is why the decision to migrate instead of rewrite is a such a good one.

So, after making that decision, which hopefully you do early, or before you are in the middle of a rewrite (as we did), you can get started. There are a few options, and I won’t go into complete detail on them, but just go over them high level..

1. Use the Visual Studio Migration tool and convert the code yourself and go at it alone

2. Look at the various companies out there that do migrations and pick one, and create a project that way.

We went with #2. After some research it was pretty obvious who the main player was out there, ArtInSoft. After researching them and watching their webcasts and reading and doing as much looking into as we could, we contacted them.

Their model is pretty simple, they send someone out to you for a week (on your dime) and they analyze the VB6 project to see how much it would take to migrate.

What I learned shortly after talking with ArtInSoft and bringing one of their analysts here is that ArtInSoft actually created the VB6->.NET conversion wizard in Visual Studio. What it is, is a lightweight version of the VBUC (Visual Basic Upgrade Companion) that they licensed to Microsoft. The VBUC in Visual Studio is very limited though. It will only migrate to VB.NET and has many other limitations, but it does still work. If you had a very small project (lines of code, complexity) – you could easily use the VBUC in Visual Studio, but most projects aren’t simple and have many lines of code, that is where you would want to use ArtInSoft and their “enterprise” version of the VBUC, as well as assistance from ArtInSoft.

Back to analysis. What they do is actually migrate your project, find any ancillary projects you might have and make sure everything gets converted. Their VBUC puts in EWI’s, Errors, Warnings and Issues, within comments in the newly converted code.

They then go throw the code and do find’s on all the EWI’s, dump them and parse them out, put into excel and pivot them to analyze how many of each EWI you might have. That way they can estimate how much effort/time/money it might be to do your project. They have done this many times, so they have it pretty much down to a science.

An example would be, you have 2500 warnings for warning XYZ – we know it takes 2 minutes to fix one of those, and they are usually all unique, so then we know the total time it would take to resolve them all.

After doing their analysis, they give you a project estimation, and then it is up to you to decide if you want to move forward, and when you would want to move forward..

 

To Be Continued…

Looking to Hire…

I have been managing two different groups @ Trek -  (Business Intelligence and .NET Software Development) for a while now, and we have some openings we are trying to file, so that is why I am putting this out here.

First role, looking to hire a Microsoft .NET Windows Forms developer, or someone with web experience looking to get into Windows Forms and eventually WPF/Silverlight, and also Windows Services. C# is the language.

Second role, looking for a Microsoft Business Intelligence Developer/DBA – SSAS/SSIS/SSRS, DBA experience preferred. Working on cubes, and ETL’s and reports and DBA stuff.

Shoot me an email at steve_novoselac@trekbikes.com with your resume and info.

Facebook Graph API – Getting Friends and Gender in C#

I recently blogged about the Facebook Graph API and if you have the Facebook C# SDK you can start making applications.

After I had my Facebook app set up, I started making a C# Console application to just get my friends and see what I could do. Here is a snippet to get my friends and their gender.

       string token = ;

            Facebook.FacebookAPI api = new Facebook.FacebookAPI(token);

            JSONObject f = api.Get("/me/friends");

            KeyValuePair friends = f.Dictionary.ElementAt(0);


            for (int i = 0; i < friends.Value.Array.Count(); i++)
            {
   
                Console.WriteLine("Friend #" + i.ToString());

                JSONObject friend = api.Get("/" + friends.Value.Array[i].Dictionary["id"].String);

                Console.WriteLine(friend.Dictionary["name"].String);

                try
                {
                    Console.WriteLine(friend.Dictionary["gender"].String);
                }
                catch(System.Collections.Generic.KeyNotFoundException knfe)
                {
                    Console.WriteLine("No Gender Specified");
                }

                Console.WriteLine();
            }


            Console.ReadLine();

There is a probably a better way to do this, but getting the JSONObject back and then getting the values you get back from that, I just kind of brute forced it. Also, handling friends that don’t have information set, the quick and dirty way was to just catch the exception. I know there has to be a better way but for now it works.