Tag Archives: Development

Facebook Graph API – Getting Friends and Gender in C#

I recently blogged about the Facebook Graph API and if you have the Facebook C# SDK you can start making applications.

After I had my Facebook app set up, I started making a C# Console application to just get my friends and see what I could do. Here is a snippet to get my friends and their gender.

       string token = <YOUR TOKEN HERE>;
 
            Facebook.FacebookAPI api = new Facebook.FacebookAPI(token);
 
            JSONObject f = api.Get("/me/friends");
 
            KeyValuePair<String, JSONObject> friends = f.Dictionary.ElementAt(0);
 
 
            for (int i = 0; i < friends.Value.Array.Count(); i++)
            {
 
                Console.WriteLine("Friend #" + i.ToString());
 
                JSONObject friend = api.Get("/" + friends.Value.Array[i].Dictionary["id"].String);
 
                Console.WriteLine(friend.Dictionary["name"].String);
 
                try
                {
                    Console.WriteLine(friend.Dictionary["gender"].String);
                }
                catch(System.Collections.Generic.KeyNotFoundException knfe)
                {
                    Console.WriteLine("No Gender Specified");
                }
 
                Console.WriteLine();
            }
 
 
            Console.ReadLine();

There is a probably a better way to do this, but getting the JSONObject back and then getting the values you get back from that, I just kind of brute forced it. Also, handling friends that don’t have information set, the quick and dirty way was to just catch the exception. I know there has to be a better way but for now it works.



Microsoft Silverlight PivotViewer: Getting Started and Business Case

I have been reading about Microsoft’s PivotViewer lately, and decided to try to get it going for myself. What is PivotViewer? Think of it as visual data slicing through a web page.

What you do is take some data, and then tie records to images, and then publish out your “collection” and you can consume it via a webpage using the Silverlight PivotViewer control. One awesome example of this is here http://netflixpivot.cloudapp.net/. But what I have been really trying to wrap my head around is how to use the as a “business” tool. Because, it is easily technically doable, but you have to have a *reason* to do it.

Working with widgets and customers and locations – what do you do? There are two things I could think of quickly. One – peruse your “master data” very fast and visually. The other is looking at some kind of metrics for your widgets, or logos of your customers you might sell too, or ? ..Well, you could..

  • Master Data/Catalog
  • Show pictures of your widgets, and create filters (they call them facets) for things like size, color, weight, model, etc. You have “one” of each and you just want to see what you offer. Almost could be a pretty sweet online catalog browser

  • Sales/Metrics
  • Do the same as a master data catalog but allow filtering by some kind of metric. Shipped items over a given time or something.

  • Something Else I Haven’t Thought Of?

Anyways, the first thing you should do before anything is get some kind of data feed. Run a query, get some data from somewhere. Start small to test. 500-1000 records.

Then the fun begins. Starting from absolute scratch..

Ok, yeah, tons of setup. Biggest thing is in IIS you need to set some MIME types: .cxml, .dzi and .dzc need to be “text/xml”

Once you have all that setup, you can do 2 things.. create your collection, and create your app. Create a blank silverlight project first:

Once you have that, there isn’t a ton you have to do to get things going with PivotViewer.

  1. MainPage.xaml
  2. Add in your MainPage.xaml, a namespace line for Pivot, and add the control

    Your end MainPage.xaml should look like this:

    <UserControl x:Class="SilverlightApplication1.MainPage"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:System.Windows.Pivot;assembly=System.Windows.Pivot"
        mc:Ignorable="d" d:DesignHeight="300" d:DesignWidth="400">
     
        <Grid x:Name="LayoutRoot" Background="White">
            <local:PivotViewer x:Name="pvWidgets"/>
        </Grid>
     
     
    </UserControl>
  3. Reference Assemblies
  4. For good measure, just reference them all, located here: C:\Program Files\Microsoft SDKs\Silverlight\v4.0\PivotViewer\Jun10\Bin\

  5. Load Collection:
  6. pvWidgets.LoadCollection("http://localhost/SilverlightApplication1.Web/MyWidgets.cxml", null);

Note, you have to make your web part of your project IIS based instead of the build in web browser. Why? Because the .cxml HAS to be hosted on a web server, it just works that way.

Now, you need to create your collection.. you can use a cmd line tool they offer on the PivotViewer site, they also have a c# library for automating things, but it is best to first just do it manually. So I used the tool they have as an add on for excel. It adds a nice little “Pivot Collections” tab

You can use this to put some data in, you probably want to add more columns than what they give you by default. For my test I just used the same image for all records to get started. I have a feeling that the biggest barrier to entry to corporate BI teams and developers is going to be the imagery. You usually don’t have someone on your BI Team that knows how to use photoshop well and do all the high res imagery, so you are handcuffed there.

For testing sake, I Published my collection to the root of my website, with the name “MyWidgets”.

I loaded up my webpage, and I can slice and dice by all my columns I had in my collection, visually.. pretty dang awesome. (Note, I just made some fake data based on attributes I am used to seeing and with an image of a bike to see what it would look like – the goal was figuring out how would this work in conjunction with current BI offerings (cubes/Pivot Tables, SSRS, PowerPivot, etc))

Now, if you think where you could take this. Each “image” is clickable and brings up the image right in front of you. You could have all the specs of that widget there, and a link to “buy”, or deeper analytics for that widget.

Some other things I found out.. using the Excel tool for Pivot Collections is dog slow. Especially with a ton of records. It has to process the images for the “deep zoom” technology and it just takes a while. Like, hours.

There are tons of possibilities here with PivotViewer, both for an external website and also internal corporate business intelligence. It will give people another way to delve into the data and turn it into information.



Extreme Pair/Group Programming – The “Hive Mind”

Recently there have been a couple of bigger development/programming issues that have needed to get resolved where time was of the essence, and some kind of “short cut” was needed. Bring in the “Hive Mind”.

In Agile, there is Scrum, and XP .. pair programming. But what if you bring in 5-6 developers, put code up on the screen and say “help me”. The informal results I have seen are astounding.

What can result is in 10-15 minutes you have a problem solved, and everyone aware of the issue or code surrounding the issue.

Now, not to say you want to do this for every situation, but in some cases it can work wonders. I’d also want to note that the results I have seen could be unique – as always, your mileage may vary.

But the Hive Mind really does bring to light the power of many heads working together to collaborate and come up with a solution quickly. Very cool.

As usual though – pretty much common sense – more open communication and information sharing leads to a better result.



SQL Schema Source Control (CodePlex)

Source Control. In my eyes, one of the best inventions of development.

Software Developers have used it for years, and it allows them to easily develop in a team environment, and be less scared they will lose a change or not be able to see things they did historically.

I have blogged a few times about source control before..

SQL Server Schema Automatic Revision History using DDL Triggers and SVN
MSFT BI In a Team Environment
Visual Source Safe Sucks/
Source Control at Home with Subversion

But this was mostly for code. Developers. The SQL Community has kind of been shafted with source control. Yea you can tie in VSS to Management studio, and others. I have never found one that works, and just wanted something to work in the background.

I originally started doing this with DDL triggers as my “>post in November kind of outlined. It worked, but wasn’t reliable enough for what I wanted and was too much setup.

So I did what developers usually do, I wrote an app. SQL Schema Source Control http://sqlschemasourcectrl.codeplex.com/

At first it worked with one database, one server, everything was hardcoded. And then it progressed, and now it is all configurable for multi server/multi database, etc. I decided to put it up on CodePlex because I think it can be improved and made to work with other source control providers, like TFS.

The code itself isn’t anything crazy, some file operations and SMO operations to get the DDL and then some functions to add/update/delete and commit to source control.

For info on how to get it working, check out the documentation page on CodePlex, I can also answer any questions here, or on CodePlex.

I have been using the app for a few months now and it has saved headaches, accidental deletes, wanting to see changes over time by developers, etc.

The feature I like the most is that it logs the SQL Agent job changes, so if someone changes a job, you can see the history…

Now, there are competitiros out there. Redgate (http://www.red-gate.com/products/solutions_for_sql/database_version_control.htm) and others. But I wanted something free and open, so that is why I am putting this out there. I would be perfectly fine not putting anything out and just using it, but I think (and hope) others could benefit from using this app

So if you are looking for semi-easy way to get revision control on your SQL Schemas and SQL Agent jobs, check out the app. There is minimal setup, but once you have it working, it just runs.

Note at this time it works with SVN and SQL 2008. Also the solutions is VS2010. I originally had it working with 2005, but no need on my end anymore for that, someone could easily make a version for SQL 2005. VS2008 solution could be created pretty easily as well. Have fun!



Product Review: Balsamiq Mockups

Have you ever wanted to quickly mockup software, website, iPhone app, etc? Most people do it on paper, or try something in Visio, or Word, or even Excel!!! But they just lack that feeling of what they want you to build. I was looking around on the web and stumbled upon Balsamiq Mockups

I download the Adobe Air app pretty quickly and got started, I wanted to see how quickly I could mock up a website I am working on, because there are some new pages and I want just a template, and then be able to quickly mock up what a new page would look like and let the other team members see it without having to code anything. Check this out (this is an example of my site, not the final version):

image

I did this in less than 5 minutes. I spent more time and refined it to what I wanted, and now I have a template which I can use for new pages, and also a mockup of a report page.

Check out http://mockupstogo.net/?c=1 to get more items, and see examples. You can mock up pretty much any existing site on the web or app, this software is awesome! I wish I would have had this for the last 8 years doing software development.

There was one thing I couldn’t figure out, which was a bar graph but having the bars as columns, but I am guessing I just don’t know how to rotate it, or I need to download some more mock objects.

If you are looking for some software to easily mock up new projects to clients, or to other users on your team, or whoever, check out this tool, you will find that it is easy to use and allows you to quickly mock up and communicate what you are looking for in an end result to your devs as well as other team members (there are some awesome collaboration tools they have baked in – even if someone doesn’t have balsamiq installed, they can import an XML file into a web version to view what it looks like, tweak it, and email it back to you)

Try it out, and happy mocking! :)



Microsoft Business Intelligence Development in a Team Environment

Today I received an email asking to some extent best practices on development with SQL Server Integration Studio (SSIS) and Business Intelligence Developer Studio (BIDS) in a team environment. Here is part of the email:

Me and another DBA belong to the same team, we have a SQL server with SSIS running. We use the SSIS transfer data among multiple data sources. In SQL 2000 DTS, both of us can save the package on the server and open/edit it in the enterprise manager. In SQL 2005, I can see the package on server, but can’t open it directly. We came out a solution: create a shared folder on the server called ‘SSIS Projects’, both of us can access to it. We run the ‘SQL Server Business Intelligence Development Studio’ on local PC, to open the project in that shared folder. When done with the change, save the package to the SSIS server. Now, we have more than 50 packages in a project. Problem is: it’s very slow when open a project, ‘Business Intelligence Development Studio’ tends to open/verify every single package inside a project, takes up to 10 mins and getting worse. We really miss the SQL 2000 DTS, but we have to turn to SQL 2005.

  1. Are we doing the right thing? Is there any better solution for SSIS developing in a team environment?

  2. When open a project, does ‘Business Intelligence Development Studio’ has to open/verify every package?

 

This got me thinking, and I figured instead of write an email back, it would be good info for a blog post. So here is what I think and some things I have done that have worked.

First, yes, SQL 2000 DTS allows you to just edit on the server, do more than SSIS, is just way better than SSIS. Wait, what? Well, yeah some people will say that, because it does one thing that might be a little rigmarole in SSIS, but no, SQL 2000 DTS is not better than SSIS, just wanted to clear that up.

So, the is meant to be a starting point, by no means all encompassing, and as always, YMMV.

One thing that I first thought about is this: Yeah, if BI devs and SQL devs have never really worked in a team environment, developing software, how would they know what to do, or best practices? They would just go about “making it work” until everything breaks or who know what.

 

So how to develop Microsoft Business Intelligence Solutions in a team environment?

 

1) Standardize on Versions

 

First, figure out what “versions” you are going to support, and what you are going to use, and get standardized on them. I am guessing majority of BI devs right now are on the 2005 stack. Yeah, there is still probably a bit of 2000 legacy stuff out there, and some people are now getting into the 2008 stuff, but 2005 is pretty much the norm from what I see, at least at this point.

So, 2005. Get all your dev’s on 2005 on their machine – same patch level, etc. Get BIDS up to the same level. Get BIDS helper installed everywhere. Strive to get all your ETL packages in SSIS 2005, get all your cubes to SSAS 2005, etc, etc. Come to a consensus on things like config files for SSIS, naming conventions, within your development and on disk – folder structure is key! With a smaller number of versions of things floating around, it makes it easy for anyone on the team to open up a solution and start hammering away without tons of setup.

2) Get Source Control

 

This is crucial! I have talked about source control in the past, and also about some that aren’t so great. Really it doesn’t matter what you use, I prefer SVN. I install Tortoise SVN, SVN proper (to do scripting etc if I need to using cmd line) and also purchase Visual SVN, an add on to Visual Studio that integrates with SVN. for 50 bucks you have your source control system. Visual Source Safe works but is outdated, honestly I hate it. Team Foundation Server is good, but expensive. Other solutions might be using something like GIT, etc. Whatever you do, just get a source control system going, and learn it well. Learn how to create repos, commit, update, revert, merge, etc. Set up a user for each BI dev and make sure they commit often, and make sure they leave comments in the source control log when they commit, history is your lifeline to go back to something if you need to! Note: exclude .suo, bin, obj directories, .user files, etc. Anything that changes every time you build, open, etc, you want to exclude from source control.

 

3) Development Box

 

You now have your version standardized, and your source control setup. You can get most of your work done on your machine, but you need somewhere to test deployments, run scenarios, etc, etc. Make sure you have a comparable box to your production server. Set it up the same, same software etc. Make sure its backed up. Let all the devs know its a dev box, it can be wiped at any time for any reason if need be. It can be rebooted 5 times a day if need be. Its a dev box! But you can test and develop and tweak and change settings to your hearts content and not have to worry about breaking Mr. Executives reports.

 

4) Developing, Merging, Committing, Collaborating, Communicating.

So now you have your setup, well.. setup. Start creating stuff. SSIS Packages, ETL’s, SSAS Cubes, SSRS Reports, the whole MSFT BI Solution. This is where stuff can start to get tricky in a team environment though. SSIS/SSAS/SSRS isn’t as clean cut as something like C#/VB.NET, etc. Everything is in some form of XML behind the scenes, and with graphical based editing, you can move stuff around and it changes the files. Things like that are going to be your enemy. This is why you need to collaborate and communicate. Usually one person should be working on one project at a time. You can get really good at communicating and then in SSIS at least have multiple people working on different packages. Also in SSAS dimension editing and stuff can be done by multiple people at the same time as long as the dim is already hooked up to the cube. But you want to make sure that you communicate, “Hey, I am checking this in, you might want to do an update”, or “Is anyone working on this or are they going to? I want to modify something, and I will check it in so you all can see it”

You want to make sure you have your folder structure, and solution/project structure set up well. C:\Projects  ..  and then maybe a folder for each major project “CompanySales” and under that, “ETL”, “Cube”, “Reports” and have a solution under each with 1 project of each type. You can also have a generic SSRS solution with many projects, which might work well for you. In any case, just come up with a standard and stick to it. Trust me it will make your life easier. The question from the email above, it sounds like they have every package in one solution, one project. Sounds like it needs to be split, multiple solutions, multiple projects.

 

5) Deployment Scenarios and Strategies

Now that you have everything developed, tested, checked in, what do you do?

Personally for SSIS I like xcopy deployments. One folder on the server, not on the C drive, but another drive, lets say “E:\SSIS” under that a folder for each project. Put your dtsx and configs in the same folder. 99% of the time you are going to call the dtsx from a SQL Agent Job, and most likely you are going to run into a scenario where you need uber rights to execute it, so learn how to create a proxy/credential in SQL security so you can run the step as that. Once you have this folder and subfolders setup, you can use something like Beyond Compare to compare the folder on the server to the one you have locally that matches. Remember to copy files from the bin directory of your project after you build it, not the files directly on your project. As far as BIDS validating every package, there are workarounds out there you can do, here is one.

For SSAS, I try to lean towards using the Deployment Wizard that comes with SQL Server. You can use BIDS deployment, but if you start doing anything advanced with roles, partitions, etc, you are going to run into trouble. Take control and use the deployment wizard. I usually like to deploy, and then process manually when developing. And then later use SQL Agent or and SSIS package to actually do processing when it comes to a scheduled processing.

SSRS, I have become used to the auto deployment from Visual Studio. To really do this though, you need a project for every folder in SSRS, which can become a pain. You can always just upload the .RDL file and connection and do it manually, but if you start off right with using the deployment from BIDS, it can make your life easier.

 

So that is just a 10 minute overview of everything to kind of get started. Everything depends on your infrastructure and the way your team is setup, etc. But I think the biggest thing to take from all of the above is to standardize on things. If you standardize on as much as possible, SQL versions, setup of machines, naming conventions, layouts, design patterns, etc, everyone can do things faster and pretty soon it will start running like a well oiled machine!



How To Speed Up Visual Studio Start Up

Add /nosplash to the end of your visual studio shortcuts..

image

also you can also go to tools->options and change the startup to an empty environment to speed up open times

 

image



Big Change #3 – New Job

Recently, in May 2008, I took a position with Stratagem. Though the summer I worked for 2 places as a consultant, KHS in Waukesha, and The Dept. Of Regulation and Licensing (DRL) in Madison.

A little history. I was full time for W3i around 2 years, and then went independent for about a year, then as a consultant with Stratagem. All the different types (full, indie, and consultant) have the pros and cons, and it all depends on what you are doing, where you are, and things you are working on.

I have done .NET, Team Lead, Database Stuff, C++, Data Warehouse/BI (Business Intelligence), ERP Stuff, more .NET and everything in between. Over the past 2 years I have come to love the database stuff more and more, especially BI and Data Warehousing. It is funny, because pretty much 99% of people have no clue what “BI” is. I was just a High Tech Happy Hour at Pooleys and everyone I talked to, “What is BI?” – and this is a tech event!! Anyways, I really do like BI and wanted to move my career forward doing BI. Stratagem hired me to do BI and work on their BI stuff, but there just wasn’t work out there, and I really wasn’t going down the path I wanted to. Stratagem is a great place, great people, I just wasn’t enjoying what I have been working on. (Although I did work on a small part time project for a couple weeks at night doing some SSIS stuff, which was exactly what I wanted to be doing full time!)

Now everyone probably knows I have an iPhone and use it extensively. A couple months ago, I installed the “Career Builder” app from the app store to check it out. It uses your location based on GPS, which I thought was cool. Just for kix, I typed in “Data Warehouse” and there was a result near Madison! Sweet, but what about details? Yeah everything that hit my buttons. Microsoft, SQL Server 2005, Analysis Services, Integration Services, some .NET, etc. Awesome!

I sent in my resume, and figured I would either hear back, and get the job, or wouldn’t hear back at all, not knowing how long the position had been out there, etc. I waited and finally heard back! Sweet. A phone interview, in person interview, informal interview, and another in person interview, and I got the job.

But who is this job with you ask? Well I am proud to say that this upcoming Tuesday I will be the newest “BI Architect” at Trek Bicycle Corp. You know, “Trek”. “Trek Bikes”. The awesome bike company. The HUGE bike company. The type of bikes Lance Armstrong rides. The USA Teams ride. The bikes that have won a ton of Tour De France’s. Sweet, sweet bikes.

Trek is located in Waterloo, WI, which is about 15 miles from where I live now in Madison, and about 14 from where I am moving tomorrow in Sun Prairie, WI. It is about 4 miles from where our band practices, in Marshall, WI. It is out in countryside, on the edge of a small town. Weird how the world HQ for this company is located in such a small town.

In any event, it is a new adventure that I am very excited for. I am really looking forward to get back into BI stuff head first and work back in SQL Server!! (I have been working Oracle for the last 2 months – ugh!)

I will even start riding bikes I’m sure, and I’m betting I am the envy of all my bike nerd friends out in PDX!!!

So this is Big Change #3 out of 3. #1 was moving, #2 is the baby, and #3 is the job. Another whirlwind couple of weeks here, and then the next couple of months, but I am excited and don’t worry, you will see my blogging to continue here. Although I am twittering more (http://twitter.com/scaleovenstove) but I still want to blog a couple of times a week. And hopefully I dive into BI/Data Warehousing and have more cool stuff to blog about in that realm.

One thing, this is the first time since I graduated college where my title isn’t some kind of “Software Dev” title. When I was indie, I really didn’t have a title, even though I was doing BI, so that is different. I don’t see myself going back to being a full time dev, even though I can do .NET. I will still use programming and development as a tool with my BI work, and also just for myself or helping friends.

Go buy a Trek! Save Gas! ;)



Why isn’t there a Web 2.0 Ajax Visual Studio?

Was thinking about this today. You can now write Word docs, Excel spreadsheets, Powerpoint presentations and the like all online (Google Docs, Zoho, etc, etc). You can record video straight to websites through your webcam, you can video conference directly through the web.

Visual Studio in the Cloud:

Why can’t you code directly into the web?

I would like to see an app that lets you create a new .NET project through a web interface, reference dll’s if you need to (upload them to your “space”) and then go about creating code, Intellisense through Ajax, you hit compile, it sends it off to the server, compiles, and gives you a result. You can then browse to your exe or your new website and view the results.

No need for a bulky IDE installed on your computer, no worries about dependencies, etc. You could code C#/VB.NET on linux and a mac with no need for mono (although you couldn’t run the exe’s – it would be the most beneficial for web apps)

You could target different versions of the framework, use new features if you wanted, all that. Heck why not have the same thing for your database. mySQL already has it with phpadmin and all the other tools, you can query and do whatever you need to through the web. Where are the offerings like this for MSFT products? Maybe I just don’t know about them.

There is CodeIDE – http://www.codeide.com/ but it is limited in languages and options. I want to see more of a full fledged Visual Studio IDE in the browser. Why? Because I want to be able to fire up a computer and just go to work, no installing, no waiting, no upgrading.

I can already see it now, Adobe Air IDE’s you can run on your desktop and sync up source code to the online IDE.

One feature built right in to this “online IDE” would be source control, revision history, etc.

I might not be possible right now, but I say give it a few years, and we will see a product like this come out, and I can’t wait.



When To Code To an Interface

When to code to interfaces? In my opinion only when you have to “INTERFACE” with a 3rd party component, or some external piece you might have to interact with. Writing an interface for every concrete class seems way to redundant. It is probably easier to convert a concrete class to an interface when you need to instead of coding and Interface and Class or every entity object you want to created. What you end up doing is just duplicating code that you will never use.

Oh yeah, your UML (who even uses UML?) will look good, but its usefulness is lacking. I say write interfaces for things like File System interaction, Database interaction, some other 3rd party or external thing you need to interact with. Then you can easily swap out the backend later
if you need to.

I just don’t get writing say, and IPerson interface for a Person object. They are just going to be exactly the same. Down the road I don’t see you swapping it out for a new “Person”. Maybe but at that point, you might as well just create your IPerson and then create your APerson, BPerson that use the IPerson interface.

I guess what I am saying is follow YAGNI (You aren’t gonna need it) principle, and you will see the benefits in your code.