R and Power BI Working Together

Microsoft recently introduced R integration capabilities in its Power BI offering stack (still in beta preview at the time of writing), and it is quite impressive. In order to get started with R and Power BI you need to download and install a distribution of R – recommended Microsoft R Open (MRAN) with Intel MKL library https://mran.revolutionanalytics.com/download/, and Power BI Desktop https://powerbi.microsoft.com/en-us/desktop/.

R integration with Power BI essentially breaks down into two parts:
I) Getting R data into Power BI and/or Power BI Desktop
II) Visualizing and analyzing data using R scripts in Power BI Desktop (no Power BI portal support yet)

There are a few great articles on R / Power BI integration:
– Running R Scripts in Power BI Desktop (Beta) http://powerbi.microsoft.com/en-us/documentation/powerbi-desktop-r-scripts/
– Create Power BI visuals using R (Preview) http://powerbi.microsoft.com/en-us/documentation/powerbi-desktop-r-visuals/
– Visualizing and operationalizing R data in Power BI http://powerbi.microsoft.com/en-us/blog/visualizing-and-operationalizing-r-data-in-power-bi/

However, there are some steps that are missing in the documentation article Create Power BI visuals using R (Preview). The next 3 short paragraphs will address all the missing steps to get that guide fully working:

1. Getting R data in Power BI Desktop or Power BI – in our example will take one of the standard R sample datasets, which are also referenced in Microsoft guides – mtcars.
For the full list of available R sample sets check out: The R Datasets Package http://stat.ethz.ch/R-manual/R-devel/library/datasets/html/00Index.html
Open Power BI Desktop, select Get Data from the ribbon bar and provide the following one-liner script, when specifying R Script as a source of data:



2. Adding additional visualization packages to R – in order to add an external package to R, such as corrplot, we will use install.packages() command.
A couple of useful links:
– Available CRAN Packages By Date of Publication http://cran.r-project.org/web/packages/available_packages_by_date.html
– An Introduction to corrplot package https://cran.r-project.org/web/packages/corrplot/vignettes/corrplot-intro.html
Open R GUI environment and run the following script:

M <- cor(mtcars)
corrplot(M, method = "number")


3. Visualizing data using R script in Power BI Desktop – insert R visualization side-by-side with other standard Power BI charts for the same data.
Once you connect to external R sample data outlined in step 1, insert R chart and select all fields from mtcars dataset so they will appear in dataset in R script editor.
After you selected all fields and Power BI Desktop generated dataframe command: ‘dataset <- data.frame(hp, mpg, qsec, vs, wt, am, carb, cyl, disp, drat, gear)’, execute this script:

M <- cor(dataset)
corrplot(M, method = "number")


SQL Client Configuration

There is a great deal of MSDN articles dedicated to describing network communication protocols and how to configure client network configuration for SQL server (Client Network ConfigurationChoosing a Network Protocol). However, MSDN seems to fail to mention how to start the network utility on the client, and the fact that there are two different flavors of it (32- and 64-bit).

Here are the executable and paths which you need to launch on the client (e.g. SharePoint servers):

32-bit version:

64-bit version:

Note: SharePoint 2010/2013 servers surprisingly uses 32-bit version configuration and not 64-bit. Nonetheless, it is a good idea to enable client libraries and create aliases for both 32- and 64-bit version.


More helpful articles on the topic:

C/C++ Calling Convention

There has been a number of changes in the past couple of years in the calling conventions of C/C++ code. The processors become a lot more powerful and provide a larger number of available registers, while the traditional calling conventions relying on stack for passing parameters slowly being phased out.

Unfortunately Microsoft and Intel decided to introduce new independent keywords to address performance issues rather than extending existing conventions of __fastcall. Microsoft came up with __vectorcall starting with Visual Studio 2013, while Intel introduced __regcall with Intel C++ 14.0 Update 1, both of which similarly utilize a larger number of registers in function calls.

C/C++ Cross Compiler Compatibility and Calling Conventions

For C and C++ compatibility the only important keyword construct to keep in mind is extern “C” – if you are intending to call C++ functions from C use extern “C” in the function declaration to prevent name mangling. Use the same declaration for C functions for calling them in C++.

Stroustrup C++ FAQ: How do I call a C function from C++?  How do I call a C++ function from C?

Stack Cleanup and Variable Arguments Relationship

Caller stack cleanup generally amounts to a larger generated code base, since for each call there has to be a corresponding stack cleanup in the code that calls the function. Hence calling the same function ten times, you would have ten sections of assembler code which unwinds the stack and arguments.

However, passing variable arguments is only possible with caller stack cleanup. The reason for that is simple – your callee may simply not know how many arguments caller pushed in that stack. Hence, if you are using variable arguments compiler would have to use __cdecl (x86) or __fastcall (x64) convention.

Calling Conventions Keywords

x86 platform: __cdecl, __stdcall, __fastcall, __thiscall, __clrcall, __vectorcall, __regcall, __syscall (obsolete), __pascal (obsolete)

x64 platform: __fastcall, __vectorcall/_regcall

Name Decoration (Mangling):

C++ compilers decorates function names by adding additional information to the function name, i.e. _foo@12 for void foo(int a, int b).

Microsoft has a utility undname.exe which allows you to decode a decorated based on its signature :

Reference: Viewing Decorated Names https://msdn.microsoft.com/en-us/library/5x49w699.aspx

C:> undname.exe ?func1@a@@AAEXH@Z
of :- "?func1@a@@AAEXH@Z"
is :- "private: void __thiscall a::func1(int)"


Intel: C/C++ Calling Conventions https://software.intel.com/en-us/node/512847
Microsoft: Argument Passing and Naming Conventionshttp://msdn.microsoft.com/en-us/library/984x0h58.aspx
Wikipedia: x86 calling conventions http://en.wikipedia.org/wiki/X86_calling_conventions
Intel: _vectorcall and __regcall Demystified http://software.intel.com/sites/default/files/Vectorcall_regcall_Demystified.pdf
Microsoft: Introducing ‘Vector Calling Convention’ http://blogs.msdn.com/b/vcblog/archive/2013/07/12/introducing-vector-calling-convention.aspx
Calling conventions for different C++ compilers and operating systems http://www.agner.org/optimize/calling_conventions.pdf
Calling Conventions Demystified http://www.codeproject.com/Articles/1388/Calling-Conventions-Demystified
Wikipedia: x86 Disassembly/Calling Conventions http://en.wikibooks.org/wiki/X86_Disassembly/Calling_Conventions

SharePoint 2013 Navigation and Mobile Devices

SharePoint 2010 and 2013 navigation unfortunately has never been designed as mobile- or touch-friendly. Top menu’s “mouse hover” effect provides serious interaction challenges when your submenus also happened to be URLs as well (catalog-based publishing sites for example). There has been a number of solutions posted to solve this issue for SP2010 sites, in particular a popular post OOTB Navigation fix to support IPad & I-Devices. However, with the user interface changes in SP2013 those solutions stopped working.

Below is a JavaScript snippet that solves the navigation troubles in SP2013, by allowing users click once to expand the submenu, and clicking it again to navigate to the underlying URL.
Insert this code at the end of the masterpage, or add it in $(document).ready() event handler:

if (navigator.userAgent.indexOf('Mobi') >= 0) {
    $('ul.root li.dynamic-children a.dynamic-children').bind('touchstart', function(e) {
        var isHoveredSubmenu = $(this).parent().hasClass('hover');
        if (!isHoveredSubmenu) {
            $(this).click(function(c) { c.preventDefault(); });
        } else {
            var location = $(this).attr('href');
            window.location = location;

A few notes about the code:
– The code only runs on mobile devices, the check for which is simplified and based on this recommendation: MDN: Mobile, Tablet or Desktop. Essentially the code searches for ‘Mobi’ string in the user agent string. This way it includes all Chrome, Safari and Opera browsers for all iOS and Android devices.
– Selector is much shorter and applies to all ‘a’ elements that have children based on ‘dynamic-children’ class presence.
– The rationale behind detecting if submenu is displayed or not, is based on the fact that when the submenu is displayed its class name is appended with ‘hover’. So we simply need to see if that class name is present in the element.

Backup, Restore, Migration in SharePoint 2013

SharePoint 2010 to SharePoint 2013 Migration
Migration process of SharePoint 2010 sites to 2013 is relatively straightforward, but requires some preliminary work done in SharePoint 2010 environment:

  • Prepare SharePoint 2010 sites for back up – put backed up sites in read-only mode or disconnect the sites from users for the duration of backup.
  • Backup the content database on SharePoint 2010 SQL server, using SQL tools
    Please note: You cannot use backup commands of stsadm or Powershell to take backups of SP2010 sites and can only use SQL Server capabilities.
    Because of this limitation if you like to upgrade only selected few site collections you might have isolate them into a separate content database.
    Here is a script you might find helpful for this isolation in SP2010:
    Get-SPSite -ContentDatabase SourceContentDbName | Move-SPSite -DestinationDatabase DestinationContentDbName
  • Prepare SharePoint 2013 environment – deploy any features or webparts used by backed up SP2010 sites if there are any. For example:
    Add-SPSolution "d:\path\customsolution.wsp"
    Install-SPSolution -Identity "customsolution.wsp" -WebApplication http://webappurl/ -GACDeployment
  • Restore SQL database backup on SharePoint 2013 SQL server, using SQL tools
  • Test the content database for any errors, by executing:
    Test-SPContentDatabase –Name contentdb –WebApplication http://webappurl/
  • Mount the content database, if previous test results are acceptable. Mounting the database will upgrade its content:
    Mount-SPContentDatabase –Name contentdb –WebApplication http://webappurl/

MS: Upgrade content databases to SharePoint 2013

Backup and Restore Site Collections in SharePoint 2013
The Powershell commands are very straightforward:

Backup-SPSite -Identity http://web/path/site -Path D:\path\site.bak –Verbose
Restore-SPSite -Identity http://web/path/site -Path D:\path\site.bak -Force –Verbose
Restore-SPSite -Identity http://web/path/site -DatabaseServer DBSERVER -DatabaseName SP_Content_Database -Path D:\Path\site.bak -Force -Verbose

If you are restoring to a specific content database, you must explicitly specify server name as well.
If you would like to control amount of site collections which can be stored in a particular content database, you can do so by using this little script.
In this script, we set maximum number of sites to 3 for the content database that contains site collection “http://web/path/site”:
Get-SPContentDatabase -Site http://web/path/site | Set-SPContentDatabase -MaxSiteCount 3 -WarningSiteCount 0

MS: Backup Site Collections
MS: Restore Site Collections

Known Issues – Backup and Restore Process Sensitivity to Environment Version

When you are backing and restoring site collections between different server environments one of the key things that you need to pay particular attention to is SharePoint patch level. The environments you are backing and restoring to must be at the same level.

Here is an example of error when you try to restore a site collection backed up in SharePoint at newer patch level.
After running the following restore command in Powershell, we see a very convoluted error message that has nothing to do with the actual error:
Restore-SPSite -Identity http://web/path/site -DatabaseServer DBSERVER -DatabaseName SP_Content_Database -Path D:\Path\site.bak -Force -Verbose
Restore Powershell
Exception: DirectoryNotFoundException – in reality has nothing to do with the actual error, which we can see in ULS log tells us that there is a version mismatch, and that is why our restore is failing.
Restore ULS Log
Blog: Backup and Restore Site Collection in SP2013

My Top 5 Freeware Tools

It has been more a year of a long online silence since I wrote the last blog post. Pursuing MBA degree while working full-time in consulting turned out to be a lot more challenging and time-consuming than I originally expected. Nonetheless, the school courses are behind now and I am finally back to the world of blogosphere. To celebrate my comeback I wanted to share some of my favorite freeware software companions that were helping me for years automating my daily tasks, while asking nothing in return:

  • TrueCrypt – in the light of the latest spying revelations and increasing malware activities this utility allows you to encrypt your confidential data and mount them as drives in your computer (this product replaces now defunct PGPDisk, which back in the 90’s paved the encryption revolution)
  • 7-Zip or WinRAR – great archivers, which trump the compression quality and offer a decent AES-based archive encryption.
  • Far Manager – my all-time favorite file and ftp manager, which allows quick selective copying of vast amount of files with the powerful built-in editor. The manager is similar to Midnight Commander on Linux, but optimized for Windows and comes in 32 and 64 bit native flavours.
  • Flexible Renamer – amazing file renamer, which can help rename your countless music or image files, using their tags or rules based on regular expressions.
  • ID3 Renamer – coupled with the Flexible Renamer, this utility allows you easily create or change ID3 tags based on the file names while using regular expressions. Perfect addition for iTunes and other music players, which heavily rely on the media tags.

And the last outside addition to that list – WinAmp – which deserves one last shout out as my all-time favorite music player. An amazingly light and versatile player, which unfortunately AOL team decided to discontinue despite its popularity and loyal customer base.

Village School

VMware Performance Enhancing Tweaks (Over-the-Counter Solutions)

I have been using VMware virtualization for quite a few years now, and even though it is generally fast and reliable virtualization platform, it does require some extra tuning to improve its performance. Here is a brief collection of tweaks that I personally use on Windows and Mac OS to improve I/O performance of VMware virtual machines. These tweaks have become particularly useful in the past couple of years, with larger amounts of available RAM in computers and desire to reduce unnecessary write operations to SSD drives, such as memory swap files.

In order to apply these tweaks you need to add the following settings either to your virtual machine .VMX file, or if you like it globally then to VMware settings.ini file. Settings.ini file is located at “C:\ProgramData\VMware\VMware Workstation\” or “C:\Documents and Settings\All Users\VMware\VMware Workstation\” folders, depending on your operating system. VMware Fusion Config files are located in “/Users/username/Library/Preferences/VMware Fusion/preferences” and “/Users/username/Library/Preferences/VMware Fusion/config” folders.

Tweak: Disable memory swap files .vmem files

mainMem.useNamedFile = "FALSE"
Unfortunately this parameter does not work for VMware Player, as it always creates virtual machine’s full memory swap file.
For VMware Fusion on Mac and Linux instead of mainMem.useNamedFile you have to set mainMem.backing flag.
mainMem.backing = "swap"

Tweak: Choose the right disk controller and specify SSD

Instead of the latest SATA AHCI controller choose LSI Logic SAS controller with SCSI disk for Windows guest OS, or PVSCSI for other types of OS. Unfortunately SATA AHCI on VMware has the lowest performance out of the three controllers and highest CPU overhead (see the references on the topic at the end). In addition to choosing the right controller, if your host disk is SSD you can explicitly specify the disk type as SSD to guest OS.
scsi0:0.virtualSSD = 1

Tweak: Disable log files for VM

logging = "FALSE"
Alternatively you can specify a different location to store the log file, if you ever need them:
log.filename = "c:\Windows\Temp\vm1.log"

Tweak: Other Disk & Memory I/O Performance Optimization

Disable memory trimming:
MemTrimRate = "0"
Disable page sharing:
sched.mem.pshare.enable = "FALSE"
Disable scale down of memory allocation:
MemAllowAutoScaleDown = "FALSE"
Memory allocation configuration:
prefvmx.useRecommendedLockedMemSize = "TRUE"
prefvmx.minVmMemPct = "100" (optional)

Tweak: Disabling Snapshots

Disable snapshots if you are not using them and prefer full backups:
snapshot.disabled = "TRUE"

Tweak: Disable Unity Mode

Unity might be a great feature for running virtual desktops operating systems, but it is not the most useful for virtualizing server OS. An annoying sign of enabled unity is GuestAppsCache or caches folder with a large number of files and subfolders. In order to disable it for your VM add the following lines:
isolation.tools.unity.disable = "TRUE"
unity.allowCompositingInGuest = "FALSE"
unity.enableLaunchMenu = "FALSE"
unity.showBadges = "FALSE"
unity.showBorders = "FALSE"
unity.wasCapable = "FALSE"

Tweak: MacBook Retina Display Optimization

Disable automatic fitting to window:
pref.autoFitGuestToWindow = "FALSE"
Set a different resolution (or in user interface select VM – Settings – Display – uncheck ‘Use full resolution for Retina display’):
gui.applyHostDisplayScalingToGuest = "FALSE"
When in full-screen mode, there is a couple of options to control how guest is rendered:
pref.autoFitFullScreen = "stretchGuestToHost" or pref.autoFitFullScreen = "fitHostToGuest"


Moving SharePoint 2010 Log and Index Locations

There are two hard-drive locations on a SharePoint server (with enabled search indexing) that are in constant use and often require significant space – log files and index files folders.

Log files location can be easily specified in Central Administration: Monitoring – Configure Diagnostic Logging.
In the Diagnostic Logging screen, you can specify a new path for trace log files and change other settings, such as how long to keep trace files, and what sort of information will be logged there.

Index files location, unfortunately, is not available through Central Administration. It is typically specified during SharePoint installation phase, along with the software installation folder. However, there is a way to change it through stsadm command:
stsadm -o spsearch -indexlocation “d:\SharePoint\Data”

You can verify that it changed successfully by running the following command before and after changing the index:


Configure diagnostic logging (SharePoint Server 2010)

Changing Index locations in SharePoint 2010


I wanted to thank Tracy Sterling for submitting a very useful tip for modifying index locations through Central Administration, when you have farm SharePoint deployments and can alter search topology.

CA > Application Management > Manage service applications > Search Service Application > Search Application Topology (Modify) > Index Partition/Query Component (Edit Properties). Repeat for each query component within your farm.

Unfortunately, all the stand-alone deployments and foundation editions would still have to go through aforementioned stsadm command. Below is an illustration of Search Service Application and its missing modify option for a stand-alone installation of enterprise SP2010 server, running on my laptop.

Moving Browser Cache Folder in IE, Firefox, Chrome, Opera, and Safari

Having recently replaced OS partition hard-drives with solid-state drives on my desktop and two laptops, I decided to tweak a few settings on my Windows in order to reduce the number of unnecessary write operations to SSD. Amongst the candidates for the move from more expensive SSD space to spinning hard-drives, I selected temporary Windows folder and browsers cache folders. Moving temporary Windows folder turned out to be one step operation – changing Environment variables in Computer – Advanced system settings dialog.

However, after this quick success moving my browsers cache away from SSD turned out to be not the most trivial task due to a large number of different browsers and their very different approach to storing configuration settings. Here is my experience and instructions on how to do it in various browsers.

Internet Explorer:

IE offers the easiest and most convenient way to change a new folder for cache.

Tools – Internet Options – under ‘Browsing History’ section choose ‘Settings’
Click ‘Move folder…’ and provide the new location, e.g. d:\Temp\IE

Heads up: After providing a new temp folder and clicking OK to save new settings, IE/Windows forcefully closes current user session, and logs the user off.


In the URL type: about:config
Create a new string preference: browser.cache.disk.parent_directory
and specify a path to your cache folder, e.g.: d:\Temp\Firefox

Mozilla Zine: Browser.cache.disk.parent directory


The only way to change location of the cache folder is through the command prompt arguments (UPDATED):

C:\…\Application\chrome.exe --user-data-dir=d:\Temp\Chrome
Chrome deprecated previous disk-cache-dir command, and the only way to move cache folder is to do it along with the rest of profile information. Reference: User Data Directory.

C:\…\Application\chrome.exe --disk-cache-dir=”d:\Temp\Chrome” --disk-cache-size=75000000
Where --disk-cache-dir points to your custom cache folder, and –disk-cache-size specifies the maximum cache size in bytes.

Here is a full list of Chrome’s command line switcheshttp://src.chromium.org/svn/trunk/src/chrome/common/chrome_switches.cc. Surprisingly, source code seems to be the only official documentation from Google. A brief blog post on the same topic: http://peter.sh/experiments/chromium-command-line-switches/.

Another useful command for Chrome, which reduces its memory footprint: --process-per-site. This command allows Chrome to run tabs from the same domain in the same process, thus reducing number of spun out processes and memory use, while still offering process isolation. http://www.chromium.org/developers/design-documents/process-models.

Even though Chrome has configuration page similar to FireFox (type about:flags in the browser URL), it does not allow changing its cache configuration.

How to Turn On/ Disable GPU Hardware Acceleration on Google Chrome.


Review the location of the current cache folder by typing in the URL: opera:about or from Menu – Help – About Opera.

To modify the settings type the URL: opera:config

Navigate to “User Prefs” tab and modify its cache folder: Cache Directory4 =d:\Temp\Opera\
(alternative quick jump link: opera:config#UserPrefs|CacheDirectory4)

Additionally you can modify temporary download folder for Opera as well, through another key:
Temporary Download Directory = d:\Temp\Opera

Opera’s Settings File Explained

UPDATE: The latest versions of Opera do not provide that user input functionality, but instead you have to use command line parameter: –disk-cache-dir=d:\path


Safari unfortunately does not allow changing the location of its cache directory, just like it does not allow changing any of its advanced settings in a comprehensive way.

about URI scheme

MBA Admission Lessons

I hardly ever write about my personal life, but this time I feel like sharing one of my recent experiences. Last week I got accepted into the MBA program at Rotman School of Management (University of Toronto), one of the top Canadian and highly-ranked international business schools.

For the past few years, I kept entertaining the idea of going back to school for a graduate business degree to pursue the almost magical MBA. However, there always seem to be something making me postpone this decision – demanding work projects and new contracting opportunities, time and financial constraints, relationship status and travel plans, long term and short term goals. Nonetheless in the end of year 2010 I came to the realization that this is the time. There always will be obstacles and unfavorable circumstances bringing a lot uncertainty and risk to any decision, and it is never going to be any easier. However, our life is never a smooth sail in a sea of calmness, but rather a journey where inner peace is the one that matters.

Admission process is always one of the first major tests on the way of getting the desired degree. In addition to taking the GMAT and writing essays, the process involves a lot of ground work, getting admission application in one piece. Here are just a few lessons I learned from the admission process:

Do your research about the business school. Make sure that the school you are applying to is a good fit with your goals and expectations. If the school is located in another city and this is going to add two hour commute to your full-time job, it is going to be very difficult to motivate yourself to go to classes every day, especially if the school was not on top of your list, but a third or fourth choice.

Attend info sessions and talk to students and admission staff. Not only you can learn more about the program and school, but you can get invaluable advice on what is exactly expected in those five essays that you need to write, and maybe get a chance to review your application with one of the admission officers before the final submission.

Don’t focus only on the GMAT score. Business schools are looking at your application to be well rounded, and pay attention to numerous other factors, such as your previous academic record, work experience, essays, and your social activity outside of work.

Find the right referees. Rather than going after high level executives, who hardly have time to reply all their piling up emails, not to mention spend time in the evening writing a proper reference about you, ask your direct manager or a client to provide a reference. Quality of his or her reference would mean a lot more than their senior executive title.

Start working on your essays early. Not only writing essays would be a good practice for AWA portion of GMAT, but also students often don’t realize the amount of time and effort that goes into well-written essays, leaving them for the last minute. Generally you would want to have at least a few revisions and get someone’s feedback on your essays before submitting them. That may add up to five or more days of work/critique per essay, depending on the quality level and expectations that you set for them.

Links to a few of my favorite GMAT and MBA admission articles:

Is an MBA a Plus or a Minus in the Startup World?
General GMAT Books Review
Quantitative GMAT Books Review
Verbal GMAT Books Review
Analyzing an AWA Argument
Ace the Essays? No, Thanks!
The Top 5 MBA Admissions Myths
How to Approach Business School Information Sessions