Serious about Office 365? Get Serious about Azure

Continuing the theme I started in “Odds are, your organization is doing Office 365 wrong” let’s talk about the relationship between Office 365 and Azure.

Or more to the point, the relationship you should have with Azure if your career is focused on Office 365.

As I stated in that earlier article, there’s a tendency for companies to fixate upon one or another element of Office 365 (Exchange or SharePoint Online, for example) and miss the SaaS forest for the trees.

To be sure, this is a missed opportunity (and may prevent you from future-proofing your career if you’re pigeonholed) but there is another, even deeper level of missed chances: failing to take advantage of Azure.

Office 365, properly understood, is an Azure application and just as Exchange or SharePoint are the “gateway drug to the cloud” for many companies, Office 365 should serve as the bridge to Azure.

When you use PowerShell to manage the identities of your tenant accounts with the Microsoft Online Services module you are connecting to Azure AD, where those identities ‘live’.

When you log into portal.azure.com using your Office 365 global admin account, you’ll see the Azure instance that forms the foundation of your tenant.

And when you log into that portal, you’ll see the full range of Azure services available to you when you activate a subscription or enter an enterprise agreement.

If your organization is using Office 365 but has not fully embraced Azure (or is unaware that it’s even possible), now is the time to deepen your commitment to cloud technology (and mature your skill set) by advocating and making the business case.

Odds are: your organization is doing Office 365 wrong

blind_monks_examining_an_elephant

Consider the very old parable of a group of blind men examining an elephant, as described by Wikipedia:

“The story of the blind men and an elephant originated in the Indian subcontinent from where it has widely diffused. It is a story of a group of blind men (or men in the dark) who touch an elephant to learn what it is like. Each one feels a different part, but only one part, such as the side or the tusk. They then compare notes and learn that they are in complete disagreement.”

The point, of course, is that without a complete picture of the elephant, each man formed strong opinions based entirely upon their limited, in this case tactile, experience.

Which brings me to Office 365 as a SaaS platform.

Although nearly all of the cloud industry excitement (and declarations of disruption) is generated by, for example, containers, serverless computing and blockchain, the truth is that for most enterprises at the moment, these are still distant – though very bright – stars. SaaS offerings such as Office 365 are penetrating the enterprise at a more rapid rate and offer transformative opportunities which, I’ve observed, are often lost for lack of vision.

For many companies, Exchange Online is the gateway drug to the cloud (along with Salesforce). And unfortunately, because of that ‘messaging’ fixation, this is where the platform stays in the minds of both technologists and decision makers. For others, it’s OneDrive for Business or SharePoint Online that act as motivators.

In each case, whatever service acts as the ‘gateway’, imaginations, and plans, tend to stay stuck in a single gear seeing that service as the whole instead of merely a part.

What’s being missed is the potential for enterprise workflow transformation, built on Office 365, that fulfills the cloud’s promise of (among other principles) speed, agility and scalability.

If you’re unfamiliar with Microsoft Graph, you don’t understand the service’s true depth, machine learning elements and full potential for developers.

If you don’t know about eDiscovery in Office 365, you’re missing an opportunity to gain greater command and control over your data.

If you aren’t encouraging end-users to embrace mobile apps such as SharePoint‘s, you’re missing a chance to increase their speed of access to data.

If you aren’t leveraging Delve, (or worse, if you think it’s something your organization doesn’t need) you’re missing a truly powerful tool that can re-architect your org’s relationship to information.

Office 365 adoption has been growing at a brisk pace. This is good news for Microsoft and everyone who works with their cloud technology stack. That growth however, does not mean the platform reaches its full potential within the organizations that adopt it. Words like “migration” and “Exchange” show up prominently in recruiter descriptions while the actual keyword, ‘collaboration‘, almost never does.

This is more a problem of breaking old habits of thought developed over years of siloed experience with on-premises, enterprise IT (with all of the bureaucratic baggage) than it is a technology challenge.

Let’s Talk About Office 365 Advanced Threat Protection: Part Two

Medieval-siege

Last time, we reviewed Office 365 Advanced Threat Protection (ATP), an enhancement to Exchange Online Protection focused on protecting end-users from two categories of email delivered threats: zero day compromised attachments and malicious URLs.

Specifically, we discussed ATP’s safe attachments policies, reporting capabilities and the labor-intensive workflow required to confirm that what ATP is telling you is, in fact accurate (i.e., that the attachment is actually malware).

This time, we’ll review ATP’s safe links, which analyzes URLs before you click through, and the PowerShell cmdlets associated with Advanced Threat Protection.

To configure ATP’s safe links, you can go to the Office 365 ECP:

https://outlook.office365.com/ecp

From the menu on the left-hand side of the Exchange admin center interface, choose “advanced threats” –

Exchange-admin-advanced-threats

 

 

 

 

 

 

From the ATP sub-menu, choose “safelinks
ATP-Safelinks

 

 

In our example, we’re using the default Safe Links Policy; to edit its properties, click the pencil icon (a common design theme in the Office 365 admin interface):

ATP-safelinks-3

 

 

 

The first setting option is “general” –

Safelinks-general-setting

 

 

 

Note that both the Name and Description fields are labels and can be changed.

The real action starts with “settings” –

Safelinks-settings-detail

 

 

 

 

Let’s walk through the options you see above.

Configuration Options

  • On or Off are self-explanatory
  • When the setting is On, potentially malicious URLs are rewritten to https://na01/safelinks.protection.outlook.com/?=url to analyze here).
  • If “Do not track user clicks” is selected, ATP will not record user click-through attempts (which means no reporting data on this action)
  • If “Do not allow users to click through to original URL” is selected, end-users will not be able to reach the original URL via the link embedded within their email

Above I mentioned that if the “Do not track user clicks” option is selected, ATP won’t gather reporting data.  But what if it is selected?

Reporting

To access safe links reporting, choose the “mail flow” option from the Exchange Admin Center interface:

safe-links-reporting

This works the same as other Office 365 mail flow reporting widgets. As you can see, you can choose the date and time range for your output.  You can also choose to search for an individual’s results by using the ‘recipient’ option (not shown but on the page if you look towards the bottom). It’s also possible to search for a specific URL.

safelinks-search-by-person-or-url

To demonstrate a search, let’s click ‘search” (not shown in the screenshot but at the bottom of the interface, as usual) and gather a broad report:

safelinks-link-listing

The recipient addresses have been obscured for obvious reasons.

By clicking on one of the line items, it’s possible to view greater detail:

safelinks-detail

This gives you the ability to analyze safe links’ actions more closely for a particular user.

I know that I mentioned PowerShell cmdlets at the top; we’ll tackle that in the next post.

Let’s talk about Office 365 Advanced Threat Protection


Warning: Illegal string offset 'width' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'height' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'autoplay' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'theme' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'loop_video' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'enable_fullscreen' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'show_title' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'show_youtube_icon' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'show_annotations' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'show_progress_bar_color' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'autohide_parameters' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'set_initial_volume' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Cannot assign an empty string to a string offset in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'initial_volume' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'disable_keyboard' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'set_initial_volume' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 66

Warning: Illegal string offset 'set_initial_volume' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 66

Warning: Illegal string offset 'enable_fullscreen' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 68

Warning: Illegal string offset 'autoplay' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 71

Warning: Illegal string offset 'loop_video' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 73

Warning: Illegal string offset 'enable_fullscreen' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 74

Warning: Illegal string offset 'show_title' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 75

Warning: Illegal string offset 'show_youtube_icon' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 76

Warning: Illegal string offset 'show_annotations' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 77

Warning: Illegal string offset 'show_progress_bar_color' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 78

Warning: Illegal string offset 'autohide_parameters' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 79

Warning: Illegal string offset 'disable_keyboard' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 80

Warning: Illegal string offset 'width' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 88

Warning: Illegal string offset 'height' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 88

english-castle-on-the-waterIn April of 2015, Microsoft announced Advanced Threat Protection (or, ATP), an enhancement to Exchange Online Protection focused on the analysis of known and ‘zero day’ threats contained in email attachments and an interception method to prevent click-through to compromised links.

Here’s a ‘Microsoft Mechanics’ video, explaining the technical details of ATP:

And here’s information about how to get ATP.

So far, so good.  But what is it like to deploy ATP in your Office 365 tenant and configure it?

 

Let’s take a look.

ATP is divided into two, distinct categories of action:

  • Safe Attachments (which, as the name implies, is the attachment analysis component)
  • Safe Links (which analyzes links against a list of known bad URLs)

Each area’s actions are configured by ATP policies which you can explore here.

In my experience thus far, the safe links component has proven to be quite aggressive, unhelpfully redirecting benign URLs (when Safelinks is active, URLs are parsed through https://na01/safelinks.protection.outlook.com/?=url to analyze here).

Your outcomes may vary, but within my tenant, Safelinks has been more intrusive than useful and has, therefore, been deactivated (no doubt, that decision will be revisited after further testing).

Safe attachments, on the other hand, has proven to be more effective, preventing 10 zero day threats from reaching end-users in a 5 day period, which is impressive needle-in-a-haystack finding when you consider the many tens of thousands of emails reviewed during that period.

There are additional steps required, I should mention, to determine what action ATP Safe attachments has taken.

Through the O365 admin GUI – Exchange Admin center https://outlook.office365.com/ecp  “advanced threats”  — “safe attachments” ATP does provide a decent visual overview of its activities:

To generate a report, click the icon that resembles a bar graph:

Screen Shot 2016-07-31 at 4.10.51 PM

 

 

 

 

By choosing “Advanced Threat by Disposition” you’ll see a bar chart reporting interface:

Screen Shot 2016-07-31 at 4.17.08 PM

 

 

 

 

 

Next, by choosing the “view pending or completed requests” link (not shown above), you’ll see a listing of the message trace activity that lies behind the reports you see in visual form:

 

ATP-message-trace

 

 

 

 

Now we arrive at a key part of the ATP process – confirming, via reporting, that the ‘hits’ are, in fact, malware.

Earlier, I mentioned that in 5 days, ATP had successfully intercepted 10 zero day threats.  How do I know that?

The answer, unfortunately, isn’t straightforward.

Let’s return to the ATP safe attachments interface:

Screen Shot 2016-07-31 at 4.30.09 PM

 

 

 

 

ATP’s actions flow from policies you create here. By choosing the pencil icon, we can take a look at the configuration for safe attachments:

Screen Shot 2016-07-31 at 4.34.24 PM

 

 

 

And, by selecting “settings” you can configure how ATP will react (or, if it will react) to suspicious attachments:

Screen Shot 2016-07-31 at 4.36.57 PM

 

 

 

 

 

In the policy shown above, ATP is directed to replace a file that’s suspected of being compromised before it reaches a recipient and redirect that file to a mailbox for further analysis.  It will also do the same when processing times out.

To get more detail explaing why ATP red-flagged a file (or if a timeout occurred) I examine what’s sent to the reporting mailbox (a shared mailbox I created for this purpose) .

Looking at the reporting account Inbox, you can see the results of an ATP safe attachments report (sender, recipient, and other details obscured for obvious reasons):

ATP-reports-message

 

 

 

 

 

Using this information, we can perform a message trace to discover why ATP intercepted this attachment:

Screen Shot 2016-07-31 at 5.36.07 PM

 

 

 

Notice the string of deferrals listed?

This means that ATP could not determine whether or not the attachment contained malware and, following policy, removed the file from the email sent to the recipient, redirecting it to our reporting mailbox.

Deferrals can prove challenging to understand since we don’t know if the attachment is compromised and, due to a current lack of detailed information regarding ATP’s performance characteristics, it’s difficult to know what makes one file an analysis hurdle (leading to deferrals) and what makes another simpler to process.

Is it file size? Or perhaps file type? Right now, we don’t know and I’ve yet to see firm information from Microsoft providing guidance (questions posed to the Office 365 Network haven’t been effectively answered).

You should be aware that either way, message delivery can be delayed by ATP:

Email delivery – If the safe attachments policy that applies to a particular recipient has an action of Block, the email will not be delivered until the attachments can be detonated by the safe attachments technology in EOP. Safe attachments will launch a unique hypervisor to open the attachment. This can result in a delivery delay of 5-30 minutes for each mail evaluated by safe attachments.”

[…]

full here

Although the quote above mentioned delays when ATP is configured to block, we’ve also seen latency when the rule is set to redirect.

Let’s take a look at a case of positively identified malware:

Screen Shot 2016-07-31 at 5.57.22 PM

 

 

Here, we see ATP safe attachments identifying malware within an attached file.

Of course, it isn’t sufficient to simply take ATP’s word for it, we need to confirm that the report is accurate.  To do that, we must submit the file for further analysis by a third party such as Virustotal or Malwr.

Reviewing the process so far…

1.) Deploy ATP to your tenant

2.) Configure the safe attachments and safe links policy

3.) Analyze the results and, in the case of safe attachments, submit those results to 3rd party tools to verify ATP’s interception.

Confirmation of findings is pretty labor-intensive and, at some critical points, very manual. It would be nice if the analytical portion was reflected in the ATP reporting interface (including a listing of deferrals vs. positive hits) and if there was a method to submit attachments for confirmation within the workflow.

So far, these options don’t exist.

In the next post, we’ll take a look at safe links and also, the PowerShell cmdlets for managing ATP. We’ll also review how to create a kind of whitelist.

The Cloud Era Demands More (and different) Things from IT Professionals

Lost-in-Space-computer-control-smaller

If you’re like me, an IT professional of ‘a certain age’, (and come to think of it, even if you’re younger but toiling in an enterprise still struggling with legacy practices) you know what it’s like to work in a siloed, IT environment.

I’m sure you know what I mean by “siloed”: the database team works separately from the SharePoint team who speak, imperfectly with the various dev teams and so on, and so on.

This approach to enterprise IT, which fosters an emphasis on individual, technical prowess over solutions, and a tendency towards isolation from the concerns and pain points of end-users and business units, is losing whatever charms it once held as cloud technologies and methodologies become standard practice.

Here’s a concrete example…

For many companies, messaging, in the form of Exchange Online, is the entry point to SaaS as represented by Office365.  Typically, the goal is to reduce server footprint, licensing costs and operational complexity by moving the email function to the cloud.

And just as typically, the messaging person, long accustomed to fulfilling that role more or less in isolation from other IT roles (with interaction, as needed with teams who need messaging services) expects to continue along that track.

But the movement of this workload to the cloud makes that nearly impossible.

Cloud services, such as Office 365, operate on a scale not achievable for most enterprises and take advantage of computing fabrics (in the case of Office 365, the Microsoft Graph) that turn discrete technologies – such as SharePoint, messaging and cloud storage, into aspects of a unified collaboration framework.

This represents a powerful change to the IT function which alters the demands placed on IT professionals:

  • Solutions: a focus on solutions over pure technical prowess
  • Flexibility: a willingness to cross technology boundaries that follow the data flow throughout your cloud platform
  • Communication: assuming an ‘evangelist’ role in your organization, promoting workflow modernization via cloud services

You find solutions by listening, seeking to mate technology to an organization’s needs instead of trying to bend people and their work process, to the constraints of a technology. In the cloud era, failure to do this leads to the use of ‘shadow’ and ‘credit card’ IT as teams work around central IT obstacles by adopting cloud technologies independently of company strategy.

You achieve flexibility by leaving your silo (dev, operations, messaging, database, etc.) and developing a broad, cross-functional body of expertise that is built on an understanding of a platform, thinking of the service in utility terms.

You develop an effective communication strategy by understanding that, a key part of your responsibility during this moment of transition from exclusively on-premises technology methods to hybrid or all-in cloud adoption, is to explain the benefits and provide guidance.

These skills have always been important, but in the cloud era, they have achieved a critical importance not seen for quite some time.  As an IT professional, your success will be measured more and more by your strength in these areas, even above your (surely solid) technical chops.

The SharePoint Online Learning Portal

SharePointOnline-logo

 

 

 

SharePoint Online is big.  Very big.

So big, that learning even its basic elements can be intimidating for end-users. So, to help organize my thoughts and provide a compendium of useful information, here is a good old fashioned link farm, devoted to SPO.

Here are the topics…(click the headings to see the instructional videos from Microsoft and other SharePoint experts)

 

  • Introduction to lists
  • How to create a List
  • Edit or delete items in a list
  • Load Excel Data Into a SharePoint Online List
  • Introduction to Libraries
  • Renaming, deleting and adding files within a library
  • Adding documents to a library
  • Understanding permissions in SharePoint
  • Managing Large Lists in SharePoint Online (greater than 5000 items)
  • Understanding the SharePoint Online Recycle Bin

 

Introduction to lists

 

“Learn what a SharePoint list is and see some examples of different types of lists, such as calendars, contacts, tasks, and custom lists.”

 

How to create a List

“This video-based training course teaches you how to create SharePoint lists using built-in apps, create and edit views of the lists, share lists with others, and set alerts so you can be notified automatically when lists change. ”

 

Edit or delete items in a list

“Learn how to edit or delete items in a list quickly in Quick Edit mode or edit the full details by opening an individual item. The video uses a Contacts list as an example, but the process is similar for most lists.”

 

Load Excel Data Into a SharePoint Online List

The easy way to turn your spreadsheet into a SharePoint Online list

 

Introduction to Libraries

“This video introduces you to SharePoint libraries and how they can help you organize documents and other files.”

 

Renaming, deleting and adding files within a library

“Working with files in a SharePoint library is easy. This short video shows you how to quickly rename, delete, and add files within a library.”

 

Adding documents to a library

“There are several ways that you can add documents to a SharePoint library. You can create a new document right inside the library, or you can upload an existing document from another location. You can also drag and drop multiple documents into a library.”

 

Understanding permissions in SharePoint

“Are you confused about how permissions work in SharePoint? Controlling access to sites, libraries and items in those libraries is an important part of using SharePoint in your organization. This conceptual video explains the basic guidelines to follow when working with SharePoint permissions. You’ll get an overview of these guidelines, including how to creating unique permissions for sites, sub-sites, and libraries by breaking permissions inheritance.”

 

Managing Large Lists in SharePoint Online (greater than 5000 items)

“No matter how big or small, lists and libraries are vital to you in many ways. But when a list or library is growing in size and might exceed 5000 items, it’s time to carefully plan and organize how the data is accessed. Why is 5,000 such a magic number? Because this is the List View Threshold, which blocks most list and library operations when this limit is exceeded. This blocking operation can be frustrating, but prevents adversely affecting the service performance of other users. Here’s some guidance for ensuring that you are not blocked, can fix the problem if you are blocked, and can stay on track. ”

 

Understanding the SharePoint Online Recycle Bin

“Unlike PC’s Recycle Bin, SharePoint Recycle Bin can store not just files and folders. It is a catch-all place for any user-created content that was deleted. That includes documents, folders, whole document libraries, SharePoint lists and even complete sites! So in other words, whether you delete a document from a document library, an event from a calendar, task from a task list, contact from a contacts list or even the whole SharePoint Site – they will all end up in SharePoint Recycle Bin.”

My Voyage to the Cloud

I wish more of my friends and colleagues in the Information Technology field would share their stories.

 

There’s a vast, hidden treasury of insight locked away in our heads – and not about technology alone but also, how organizations use and adapt to technology (or don’t).

This recently came to mind (and inspired this post) as I reviewed the last few years of my career over a few glasses of wine. During this brief time, my entire point of view about the purpose and future of IT has dramatically changed.  I’ve travelled the path from cloud skeptic to cloud enthusiast.  What transported me from one pole to the other?

That’s the story I’m going to tell.

A Sense of Dread

 

The_Scream_400

My career in Information Technology – which started well over a decade ago – was practically an accident. After leaving college, I worked in banking in a very entry level position.  It was a tedious job that involved the manual reconciliation of account data (i.e., did deposits match withdrawals? …and other minutiae).  Hour after hour of eyeballing columns of information searching for inconsistencies inspired a sense of ennui.

Wasn’t there a better way?  Wasn’t this a perfect job for software?  Surely there was an algorithm that could accomplish this.  I’d worked extensively with computational methods in college, solving statistical problems using the resources available in the computer lab so I knew there were powerful alternatives to this drudgery.

I presented my ideas to management who, with one notable exception, politely thanked me and promptly returned to their 1950s mental cocoon. Until, that is, the FDIC came along.

Mechanization Takes Command

Modern Times
Modern Times

Without going into deep detail I’ll say that when the bank was audited it received a failing grade for the lack of investment in Information Technology (among other sins).  Suddenly, there was a mandate to modernize the organization’s minimal IT infrastructure.  A VP with whom I was friendly pointed towards me and said: ‘that’s the guy who will make it happen’.  As a professor of mine often said, ‘repetition is the key to learning’ – my mantra about the need for IT, combined with a government directive and the sponsorship of a mentor had changed my career, almost overnight.

Welcome to the Present – and Future

MCQ-cloudcityThis ushered in an exciting time; network cables were laid, a data center was made, a client server infrastructure was built and methods were created to import data from offsite mainframes into on-premises servers for real-time analysis by financial personnel, all coordinated by me.  It was a whirlwind of activity that completely transformed the way the bank operated.  And yes, the account reconciliation process – tailor made for automation – left human hands and became the work of algorithms.

The Age of Consultancy

Style: "Mad Men"

Eventually, there were ‘no more worlds to conquer’ at the bank and I found myself growing restless – a not uncommon condition of people in our field.  A friend suggested I interview with a consultancy start-up he’d recently joined – a firm composed of a combination of young hotheads looking to dive into the world of client server development and older, infrastructure veterans, weary of the politics and mission silos of corporate IT.  I was impressed by this group of visionaries and made the leap.

This started the next phase of my career, defined by a sort of creative chaos as I was sent from one assignment to another with only the vaguest idea of what I was supposed to be doing.  One moment, it was writing transact SQL code and the next, it was acting as a sysadmin for a massive farm of Solaris servers.

Despite the uncertainty, I learned three valuable lessons from this time:

  • To be open minded and technology agnostic
  • To cultivate a spirit of constant learning
  • To think of myself as a technologist first and not as the champion of a particular company’s stack

These lessons would serve me well as the next chapter began.

 

The Importance of Deep Knowledge

20053772190_6606a23662_zBy now, I was comfortably operating as an IT generalist, working under the umbrella of the consulting firm whose business was growing at a rapid pace.  An encounter with a seasoned professional however, would shake my confidence in future prospects and reorient my thinking towards deeper topics.

While engaged on a lengthy project, one with a heavy emphasis on Tru64 Unix, I had the pleasure of working with a man whose knowledge of that platform was profound.  He took me under his wing, stressing one important message: ‘it’s good to have a wide range but you must possess deep knowledge in at least one area to be a serious professional.  Pick something you love and make it a part of you.  If you do that, and it’s critical to business, you’ll always excel.’

I knew what I needed to do: I would become a messaging expert.

You’ve Got Mail

microsoft-exchange-logoThis turned out to be precisely the right decision as Microsoft Exchange – once a ‘toy’ product – was coming into its own as a robust messaging platform.  Integration with Active Directory and the publishing of an API that programmatically extended the platform and broadened the amount of knowledge required to truly be considered a subject matter expert.  With the introduction of versions 2007 and above, Exchange graduated to enterprise class.  And also, the foundation for SaaS versions of the product were laid.

Messaging is the SaaS Gateway for Many Firms

Having established myself as a messaging SME focused on MS Exchange, it was only a matter of time before Office 365, the mature successor to what was once known as the Business Productivity Online Suite (or BPOS) entered my life.  My first encounter with BPOS left me cold – I was firmly rooted in the world of data centers you could touch, bare metal and virtual machines you owned and the illusion of control.

Of course, along with that supposed control there came a host of challenges that often wrecked weekends and ruined sleep: server malfunctions, active directory issues, VMWare host or VDI problems, network communication challenges, firewall configuration mysteries and on and on.

Despite this nearly constant churn of drama – even in well-designed and reasonably well behaved infrastructures – I was deaf to the potential of (then nascent) cloud technologies.

 

But all that was about to change.

Ascension

forbidden-planet (1)I accepted a position with a firm that had gone all in with AWS and Office 365: AWS on the PaaS and newly created DevOps side of the house and Office 365 on the SaaS/back office side (oh and of course, the nearly ubiquitous SalesForce SaaS was heavily in-use).  Office 365 was adopted, it was hoped, as a way to eliminate the expense and infrastructural complexity of on-premises Exchange – the theory was that less knowledge would be required to manage these cloud technologies.  Of course, this turned out to be wrong but what was discovered along the way was the scalable power, flexibility and velocity made possible by leveraging the public cloud.

My discovery was that by letting go of an attachment to legacy practices – of a fixation on ‘owning’ the infrastructure – I could explore the use of computing power as a utility and change my career direction from being part of a cost center, often beset by crises, to crafting solutions and actually being the business.

Through Office 365, I reoriented my thinking away from isolated areas (i.e., the ‘messaging’, or SharePoint, or IM silos as separate areas of expertise) and towards SaaS as a collaboration tool set that enabled the organization to become nimble.  Through AWS (and a little later, Azure) I learned to rethink my relationship to server assets from the pet to cattle model.

This has reinvigorated my career and opened an exciting new chapter.

So much so, that I’ve become an unabashed enthusiast and ‘evangelist’ for cloud technologies.

 

 

Adopting cloud technologies with both feet on the ground

Kirk_and_Spock_view_Stratos_from_surface

If a Google search is any guide (and it usually is) 2016 is turning out to be ‘the year of the cloud’ (or perhaps the real year of the cloud).  Here’s one prominent example.

While this represents an exciting opportunity for those of us who have focused our career objectives on SaaSIaaSPaaS and DevOps, there’s also the risk of misunderstanding what these technologies mean and how they should be adopted by your organization.

One of the things I’ve noticed is a tendency – among both vendors who’re excited about their products and colleagues who’re excited about new methods – to speak of powerful offerings such as Cortana Data Analytics and other thourougly 21st century creations without first concentrating on the hard work of dissecting current on-premises network, data and computational usage.

This can lead to the equivalent of inventing advanced space flight technologies without first developing airplanes  – of, in other words, trying to run before you’ve even walked.

To offer a concrete example (because I’m all about specifics), I’ve witnessed businesses attempting to adopt the most advanced cloud-based technologies such as Amazon EMR, while skipping other, more mundane, ‘keeping the lights on’ platforms which could also benefit from being hybridized or moved entirely to the cloud.

This has led me to adopt a basic cloud adoption outline:

1.) Have you considered the impact on your network?

This is at the most basic level but is often overlooked. Users and processes that consume cloud services will need robust bandwidth or perhaps even dedicated connections such as Microsoft’s ExpressRoute or Amazon’s DirectConnect to enjoy a consistent experience.

2.) Have you inventoried and analyzed your current data center portfolio to see what can be hybridized or moved completely to the cloud?

If, like most companies, you’re using Active Directory hosted by on-premises servers,  they’re probably aging and in need of upgrading (at both hardware and functional level). Maybe it’s time to consider a cloud supplement such as Azure AD or Amazon’s Directory Service instead of performing yet another data center project.

The same can no doubt be said of your on-premises database investment. The point, of course, is to start with the basics and then work your way to the exotic.

3.) Have you determined the licensing cost of upgrading your on-premises assets when compared to the usage costs of cloud services?

Years of experience and habit have trained us to upgrade hardware and software as needed, after receiving management buy-in (and usually after running against significant performance issues with strained and aging platforms).

But in recent years, there’ve been changes to the licensing model of on-premises systems that may make cloud alternatives more financially attractive at scale.

For example, Microsoft’s per core licensing may significantly increase the costs associated with a bare metal server upgrade for SQL server.  The better your server, the higher your license costs may turn out to be.

Cloud technologies offer organizations an exciting opportunity to, as Amazon says, reinvent the way computing power is used.  To truly make the most of this opportunity however, a lot of homework is necessary.

Big Data: A Learning Journey


Warning: Illegal string offset 'width' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'height' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'autoplay' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'theme' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'loop_video' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'enable_fullscreen' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'show_title' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'show_youtube_icon' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'show_annotations' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'show_progress_bar_color' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'autohide_parameters' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'set_initial_volume' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Cannot assign an empty string to a string offset in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'initial_volume' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'disable_keyboard' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 61

Warning: Illegal string offset 'set_initial_volume' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 66

Warning: Illegal string offset 'set_initial_volume' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 66

Warning: Illegal string offset 'enable_fullscreen' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 68

Warning: Illegal string offset 'autoplay' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 71

Warning: Illegal string offset 'loop_video' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 73

Warning: Illegal string offset 'enable_fullscreen' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 74

Warning: Illegal string offset 'show_title' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 75

Warning: Illegal string offset 'show_youtube_icon' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 76

Warning: Illegal string offset 'show_annotations' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 77

Warning: Illegal string offset 'show_progress_bar_color' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 78

Warning: Illegal string offset 'autohide_parameters' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 79

Warning: Illegal string offset 'disable_keyboard' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 80

Warning: Illegal string offset 'width' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 88

Warning: Illegal string offset 'height' in /homepages/17/d119946931/htdocs/blog/exchange/wp-content/plugins/youtube-video-player/fornt_end/front_end.php on line 88

Lost-in-Space-computer-control-smaller

As nearly everyone and her kindly grandmama knows by now, “big data” is, well, a big deal.

Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, and information privacy. The term often refers simply to the use of predictive analytics or other certain advanced methods to extract value from data, and seldom to a particular size of data set. Accuracy in big data may lead to more confident decision making. And better decisions can mean greater operational efficiency, cost reduction and reduced risk.”

[…]

full at Wikipedia

One of the leading technology platforms for extracting insights from very large data sets is Apache’s Hadoop.  Other heavy hitters are MongoDB, Apache Cassandra and Apache Storm.

So you’ve read the articles and watched intriguing Youtube videos (such as the one shown below) about big data and are eager to dive in.

Where do you start?

My suggestion is that you take full advantage of the learning tools made available by one of the leading Hadoop providers, Cloudera,  Hortonworks, Microsoft or Amazon.

Of course, Hadoop isn’t the final word on big data analytics but it’s a good place to start and Cloudera provides a downloadable virtual machine (along with corresponding guidance)  that can help you learn the basics.

In future posts, I’ll be walking through the learning process, detailing my experience to (hopefully) help others who’re motivated to become adept in this critical technology.

 

SharePoint Online: Copying Files to the Cloud with SPFileZilla

Brooklyn-Bridge-1950s

 

Cloud technologies offer many advantages but also pose quite a few logistical challenges.

For example, how do you move local data you’ve accumulated from your computer or on-premise network to a OneDrive for Business or SharePoint Online site collection?

(And yes, we’re all familiar with the OneDrive for Business sync client but, on the Windows side,  that can get a bit wonky if you need to upload hundreds of megabytes in the beginning and the OS X client is often practically useless…although new clients are reportedly on the way.)

There are quite a few answers to that question; here’s Microsoft’s guidance.

That’s all good but I’ve had great success with an open source tool named SPFileZilla.

Here’s a quote from the project’s page:

Inspired by FileZilla, the fantastic free FTP client, SPFileZilla allows you to navigate and manage SharePoint as if it were an FTP Server. Browse site lists, document libraries, folders, and files. Download and upload files and folders, including support for nested folders hierarchies. Create new folders, rename existing files and folders, and delete files and folders. Also, you can copy paths to files and folders to your clipboard. Drag and drop folders/files into the application to upload to SharePoint.

[…]

Full at the SPFileZilla homepage.

The key to this project’s success is its FileZilla, FTP-esque style of presentation which makes file transfer and management very simple.

Take a look at the main screen to see what I mean (and by the way, you should be a site collection owner or admin to effectively use this tool):

SPFilezilla-one

At the top of the interface, you enter the URL of your SharePoint Online site collection –  for example, https://your-domain-sharepoint .com/sites/your-site-here.

Your username and password are, of course, the same that you use to authenticate to the Office 365 portal and other cloud-linked services (such as Outlook).  To make sure the application ‘knows’ you’re trying to connect to an online site and not one within your on premise domain, check the “Is SharePoint Online?” button.

SPFilezilla-two

By clicking the “Quickconnect” button, you’re logged into your site collection:

SPFilezilla-three

Notice the right-hand side of the interface which shows the folder structure of your site collection.  By selecting files and/or folders on the left-hand side of the window, and clicking the rightward facing arrow button (after browsing to your target folder within SharePoint) you can copy files from your local drive to SharePoint Online.

That’s it; remarkably simple.

And simplicity equals elegance.