Excel Archives | Eppert Consulting

06/03/202003/03/2020

Connecting Excel to SQL Server: Adventures in an Almost Legendary Project

We were hired to convert a large UK accountancy firm’s many interconnected Excel spreadsheets into a robust SQL Server database project. Part of the project included exporting data from Excel to SQL Server.

The intricacies of doing this were immense.

The London accountancy firm

The accountancy firm is a large, respected company based in London. They manage the accounting of thousands of independent contractors across the UK, doing everything from tax returns to day-to-day payments for them.

They deal with massive amounts of data.

Using Excel as a database was no longer an option

Understandably, the company had outgrown its in-house Excel spreadsheets and systems.

We had already created a fairly sophisticated Excel database solution for them which manipulated and processed the incoming and outgoing data efficiently. But the company had reached a level of expansion where there was simply too much data for Excel to deal with speedily and rapidly.

And there were other concerns as well, such as:

Portability of the data, should the company wish to move to a different platform in the future.
Advanced reporting which was becoming ever more difficult, given the sheer quantity of data they were dealing with.
Ease of backups.
Maintaining the integrity of the data as the company grew and more staff gained access to raw figures.
Security of data.

Excel had been doing a great job until the numbers hit several million records per spreadsheet.

We had already created a database in Excel. Now we needed to take that “database” and put it into an actual database — something like MySQL or SQL Server or Oracle.

The decision to switch from Excel to SQL Server

Data abstract — Photo by fabio on Unsplash

Excel tends to “play better” with Microsoft products, so we were not keen to use MySQL.

Eppert Consulting had previously completed a gargantuan forecasting project for a German company whose preferred database was Oracle. There are several intricacies involved in connecting Excel to Oracle, more so than with SQL Server.

The accountancy firm was comfortable with Microsoft products, and so it was decided that we would connect Excel to SQL Server.

That’s when the work began.

If I were to detail for you every step of the project’s life cycle, this article would end up being a book. The entire process was both fascinating and rewarding.

Excel is a powerful tool, but combined with SQL Server as its back-end data provider, it is without a doubt the most effective tool any company can have in its arsenal. It can do everything from creating quotations to running sophisticated forecasting software which increases sales.

The project, looking back at it, feels now almost legendary in its scope, even though it did not feel this way at each step of the process.

Here are some details of what we did.

The initial data import from Excel to SQL Server

Although both Excel and SQL server have built-in tools for importing and exporting data, I would not recommend using these for a project with so much data.

The task is best done with VBA programming.

Once the data has been imported, it well behooves the programmer to spend as much time as is necessary verifying the correctness of that imported data.

This step cannot be undervalued. There is so much potential for minor importing errors which could have snowball effects later on. You will save yourself much debugging time if you verify the data fully and completely at this stage. A fresh set of eyes helps. Excel formulas and some direct VBA programming checks are also useful.

Daily importing and exporting from Excel to SQL Server and vice versa

Once the data has been imported into Excel, it needs to be pulled back into Excel in order to work with it.

Nothing beats Excel’s myriad functions for working with data. The accountancy firm was keen to move the data to SQL Server, but dropping Excel as an accountancy tool was never in question.

There are several ways to regularly export and import from SQL to Excel, and vice versa. The most common one (although we do not recommend it) is to open up a connection to the SQL Server using VBA code, pull the data, close the connection, and keep doing this for each transaction.

This is inefficient.

Increasing Excel’s speed was imperative

Long exposure of cars driving at night, suggesting speed — Photo by Marc-Olivier Jodoin on Unsplash

Speed was a key factor in deciding to move from Excel to SQL server, and constantly opening and closing connections would slow the software down due to the enormous amount of transactions this company was performing a minute.

We recommend pulling the data from SQL Server into a hidden workbook in Excel using VBA programming. Do this when the program first opens up, then periodically refresh the data to maintain consistency.

There are fine details on how this can be done without data conflicts, but that is beyond the scope of this article.

The key thing is to “flatten” the data. Excel is worked with most easily when all the data is on a single sheet. SQL Server is the exact opposite. By the very definition of a relational database, best practices in SQL Server dictate that one must separate the data into tables, and relate them one to the other. (The technical term for this is “normalising” the data.)

These two paradigms are in direct conflict with each other.

Interfacing between Excel and SQL Server

If you wish to use Excel with SQL Server as a back-end, you will need to provide an interface between the two systems which “flattens” the data so that it is mostly in one sheet in Excel. To send data to SQL Server, you need a reverse system which “normalises” it into separate tables on the way back.

This might seem like quite some programming, and it can be. But once the initial elements are in place, the code can be re-used for all data transfers between Excel and SQL Server.

Excel is the boss in an accountancy firm. It is unlikely that you’ll end up using even a tenth of SQL Server’s many features. When connecting Excel to SQL Server, the programming task is made easier in the long run by doing the majority of work in Excel’s VBA programming language. This makes future debugging easier.

Further Excel integrations

But it didn’t stop there. By the time we had completed the bulk of the Excel to SQL Server integration, the tool had reached a stage of such value that the company requested further modifications to it.

Each of the modifications was small, and took very little time, comparatively, to implement. But looking back at the completed product, I am still amazed at how much we achieved. I am also amazed at how unbelievably flexible Microsoft Excel can be when it comes to connecting and integrating it with other tools and software such as Sage, Adobe Acrobat, Amazon Web Services, etc.

Here are just a handful of the many integrations Eppert Consulting programmed into this mammoth and invaluable piece of Excel accounting software.

Integrating Excel with Outlook

One of the first requests was to integrate Excel with Outlook — preparing a client’s tax return, for example, and then sending the return to them in a PDF format using Outlook.

We did this in the Visual Basic for Applications programming language, of course. Advanced tasks such as these can only be done using Excel’s programming features.

We ended up programming many integrations between Excel and Outlook. Here are some of the features that were required for each or all of those integrations:

Correct formatting of the prepared Excel document so it looked professional when exported to PDF.
Automatically emailing it to an email address associated with the client (stored in the underlying SQL Server database).
Various email templates which could be edited by the client.
Email templates automatically selected when clicking the “Send” button, based off options filled in, in the Excel spreadsheet.
Bulk emailing of attached financial figures (either in Excel or in PDF format) utilising an underlying data store.
A sophisticated document approval process, resulting in a completed document finally being sent out by email once it had been verified and checked at various points.

Connecting Excel to bulk-emailing and SMS software

Hand using mobile phone — JÉSHOOTS at Pexels

The company uses a bulk emailing service, independent of Microsoft Outlook. It also uses a bulk SMS service which requires messages to be uploaded to a third-party company’s website in a specific format.

We programmed a feature to create these messages with one click, formatting the upload-file within Excel itself, using VBA programming code.

Importing banking data into Excel — integration with HSBC Connect, RBS and other financial institutions

Safe deposit boxes — Photo by Tim Evans on Unsplash

One of the most ambitious tasks we had was to import financial transactions from HSBC, Royal Bank of Scotland and other financial institutions into Excel.

Excel also had to export payment data directly to these financial institutions.

Each institution had its specific way of accepting data, as well its own security practices. We programmed several features so that Excel could both receive and send data for thousands of bank accounts, for which this accounting company had a mandate.

Programming Excel to work with Amazon Web Services

The company has produced several hundred-thousand documents over the years. Amazon Web Services was chosen as a secure location to store these documents.

The Excel tool needed a function where it could directly upload generated documents to Amazon Web Services’ Simple Storage servers.

To achieve this, we had to program an external tool in Microsoft’s .Net programming language, because Amazon did not provide an interface for Excel.

Well, we did it, and we also synchronised the data for the uploaded documents with our local SQL Server database so that we always had a local record with which to compare Amazon’s data to ours.

We also maintain separate backups of all documents. The Excel software takes care of placing each copy of the document — backup and original — where it belongs.

Connecting Excel to PHP, MySQL and Microsoft .Net websites

Meanwhile, web technology continued to progress at a staggering speed. The accountancy firm’s clients requested more and more that their documents be available to download online.

To do this, it was necessary to create a second online database to store information regarding each user’s documents. We had to program functions to:

Notify users of their generated documents.
Ensure that only authorised users could download documents.
Upload documents into a secure location where they could be downloaded by authenticated users.

We also programmed a feature where clients could approve or disapprove financial transactions using an online platform.

We used a MySQL database to store data because the firm already had a PHP website in place.

This eventually led to a Microsoft .Net website, an additional SQL database, on and on and on…

The project kept growing, each time adding value to the firm, and improving its ROI.

Excel can connect to anything — anything at all!

There was never any dearth of ideas, and the software kept growing. Each new feature requested was a challenge, and I am happy to say that Eppert Consulting rose to meet those challenges.

No matter the request, we found a way to achieve it. This speaks more for Excel’s versatility than to our programming skills. Although an in-depth knowledge of Excel and all its potentials is absolutely necessary to come up with solutions to unusual problems — to think “outside the box.”

03/03/202029/02/2020

4 Ways to Know if an Excel Expert is Really an Expert

Professional man walking in daylight — Photo by bruce mars on Unsplash

To properly define an Excel expert, we must know the meaning of the word expert.

ex·pert (noun): one with the special skill or knowledge representing mastery of a particular subject — Merriam Webster

“I am a self-proclaimed expert” seems to be the mantra of the modern age. Degrees and doctorates at established universities, coupled with a long track record of competent service in a given industry, were once the criteria for defining expertise. But the emergence of ambiguous fields of expertise such as Search Engine Optimization, User Experience Studies, and Social Media Analysis has blurred the meaning of the word “expert.”

Excel is a little like that. There is no university which offers a degree on “Excel Expertise.” But there are ways to tell if the Excel expert you’re about to hire really is an expert or just a self-proclaimed guru.

1. Can the purported Excel expert program in VBA-Excel’s built-in computer programming language?

Programming code on computer screen — luis gomes at Pexels

If he cannot, he is immediately not an expert. There is no need to read any further.

“VBA” stands for “Visual Basic for Applications.” It is Microsoft Excel’s “hidden” programming language. Its power and potential for solving everyday business problems with Excel cannot be understated. Professional and competent use of this feature can turn any mediocre Excel workbook into a workhorse of dazzling efficiency.

All of the programming language’s possibilities are impossible to enumerate here. Take a look at some of our customised business tools for Excel to get only an inkling of what can be done with Excel programming.

And that’s just scratching the surface.

A true Excel expert doesn’t merely dabble in Excel’s programming language. He is a master at it.

The more complex your project’s needs are, the more VBA programming will be required as a skill.

2. Can the expert demonstrate his knowledge of Excel functions and formulas quickly and easily?

Magnifying glass on maths formula — Ian Panelo at Pexels

An Excel specialist who does not have an absolute and total command of the majority of Excel’s built-in functions and formulas has no right to call himself an expert.

This one is easy to spot, even if you don’t know the formulas yourself, because a true expert should also be able to explain the formulas at a basic level to you, and in a way you can understand.

Ask questions about the potential project. Have him discuss with you how he would solve certain problems regarding the workbook. People who want only to dazzle you might leave you feeling more confused after such a conversation. A true expert will leave you feeling enlightened.

The true Excel expert knows his formulas and can explain them clearly when asked about them.

3. Are his workbooks visually appealing as well as functionally correct?

graphs on computer screen — Photo by Luke Chesser on Unsplash

Excel programming and Excel functions are the guts of the vehicle–the engine, the transmission system.

When a person first starts learning about Excel, often they get embroiled in the potential of these formulas. Their workbooks become overly complex. They sometimes want to try every single formula merely because it exists.

Your true Excel expert has been through this phase early in his career. He has fiddled with the engine, taken out his spanner and wire-cutters and pliers and toyed with everything from the spark plugs to the interior light. He knows how the car’s internals work through and through, and knows what is important and what is not.

After that, he can get into presentation.

No one wants to drive down the road in a sports car which looks like a jalopy.

The true Excel specialist will know how to make the final workbook shine. He will make it look good when it’s printed. The input cells will be easy to see. The cells with formulas in them will be protected so that they are not mistakenly overwritten.

Design is not something done as an afterthought. It is best done during the workbook’s creation. And if your Excel pro truly knows what he’s doing, the final design will be clean and simple, yet professional.

4. Does he have knowledge and experience in features seemingly unrelated to Excel?

Too many people consider Excel a tool found only in the accounting department, or in a stock broker’s arsenal.

Additionally, some people think that Excel can work only with Excel itself. In truth, Excel can work with almost all tools and software on your computer as well as on the internet.

A knowledge of Excel’s built-in programming technology is required for this. It opens vistas wide and large on Excel’s potential for small and large businesses.

At Eppert Consulting, we have programmed Excel tools which connect and work with Amazon Web Services, Dropbox, Google Maps, Microsoft Word, Adobe Acrobat Reader, Microsoft Outlook, Microsoft Access Databases, SQL and Oracle databases, and many, many other services and software. We have programmed tools which encompass PDF document manipulation, web uploads and downloads, image processing, the works.

If your expert is truly an expert, he has worked with a multitude of external services which enhance and accentuate Excel’s inherent features.

The scope, truly, is limitless.

Summary of how to choose an Excel expert for your project

Summary on paper — Photo by Glenn Carstens-Peters on Unsplash

No Excel specialist can claim expertise in Excel without an in-depth knowledge of Excel’s built-in programming language. Lack of its use is the difference between “good workbooks” and “spectacular workbooks.”

If he does not show a fluidity of understanding of Excel’s many functions, he is no expert. A true expert in Excel has such an in-depth knowledge of the subject that using it is a piece of cake for him.

A veritable Excel expert knows how to present his Excel projects. He is familiar enough with the inner working of Excel to have enough time left over to make your project also look sleek and spectacular. And the very best Excel experts do this all within budget.

Excel is not an island, and the truly competent Excel professional will know how to interlink Excel with other software and tools so that they work together. This way, he can provide a suite of functions which improve and facilitate complex tasks within your business.

Sure, there are also the obvious aspects of determining if someone really is an expert at something: Does he have good references? Does he have a wide and varied number of clients? Does his work speak for itself?

But these are things which one must determine with any business, not only Excel specialists. The above 1 – 4 are specific to Excel, and anyone claiming to be an Excel expert must pass with flying colours on each and every one of these points.

If you were to run Eppert Consulting’s skills and résumé through the above 1 – 4, I would hope you found us to be just the right Excel experts you needed for your next project.

20/02/202025/02/2020

Create Quotes in Excel Which Close the Deal

Man preparing a quote manually — Photo by Helloquence on Unsplash

There is no better software for creating quotes than Microsoft Excel. Yet the intricacies of Excel can be daunting for anyone not familiar with the program.

“It’s all about the presentation.”

It’s the simplest rule in Marketing. If you don’t “wow” your potential client, they’ll go somewhere else. And wowing the potential buyer has little to do with price.

In a study conducted in 2012, Business Weekly reported that “New research suggests that low prices can backfire for retailers since consumers sometimes see cheap prices as a sign of a cheap product.”

No, you wow your client with great value for money, unbeatable quality, and superior service. But how do you even get them to sit up and listen, to actually look at your proposal?

“New research suggests that low prices can backfire for retailers since consumers sometimes see cheap prices as a sign of a cheap product.”–Business Weekly

Too often we hear from clients that their in-house CRM or Accounting software–everything from Sage to massive SAP applications–simply doesn’t cut the grade when it comes to creating professional quotes which bring the deal home.

Either the proposal it creates looks “boring” or bland, or it simply isn’t customizable enough to make it look unique.

The other rule of Marketing: You have to stand out

Professional clients want value for money, not necessarily cheapness. To close the deal, you need to stand out from the competition with your proposal.

Your only choice is to spend hours upon hours putting together a quotation for that “One Big Client,” neglecting all the other quotes that are being demanded of you–quotes which themselves also need to be professionally done.

It’s the endless conundrum of running a successful, expanding business.

Stressed out woman sitting at laptop — Photo by energepic.com from Pexels

Creating an Excel Quoting Tool is the way to do it

We have been creating customized Excel solutions for over twenty years–everything from Day Trader Excel Tools to an Excel Payroll Tool to, yes, a sophisticated and easy-to-use Excel Quoting Tool. Nothing comes close, competitively, to what Excel can do for a business. It is a phenomenal product which only keeps getting better.

Like all “good software” out there, Excel’s true potential is only really seen when one knows how to use every part of it. And that task can be daunting.

Here at Eppert Consulting, we are constantly amazed to discover that even some accountants don’t know all of Excel’s potential tricks and tools.

An Excel Quoting Tool is the easiest thing in the world to create–if you know Excel. It can be done in only a matter of hours for a simple tool which would generate professional quotes in seconds. The quotes would be designed to your specifications, then exported to PDF or Microsoft Word.

Steps to create an Excel Quoting Tool

1. Make it easy to input data

Excel has inbuilt tools to prevent user error. The trick to fast and professional quoting in Excel is to make the user-input almost automatic.

You can lock cells so that users don’t mistakenly overwrite key formulas and values.

You can colour-code input sections–locked cells one colour (such as grey), input cells another colour (such as light green).

One spectacular feature of Excel is conditional formatting. It’s really quite difficult to overpraise this feature, especially in quoting tools. You can have a hidden sheet which compares this quote to older quotes or, say, to a median value. If you’re quoting too high or too low, the cell could suddenly turn red to warn you.

The goal is to think as little as possible when crunching the numbers, and reduce human error completely. The ideal scene is to type in your values, click a button and export to a gorgeous PDF which impresses the heck out of your potential buyer, and knocks over all of the competition

If you really want to get sophisticated, you could design a user-form to capture your data and insert it into an underlying sheet. But this is really getting advanced. (We can help you here, if you wish.)

Laptop with Excel-style dashboard open on screen

2. Design a spectacular quote template

This is where you wow your potential client. The layout of this document needs to be visually appealing. It needs to catch the eye, direct it, impress.

It can all be done in Excel.

This is not the same sheet into which you input your values!

A lot of people make this mistake. Ideally, your input screen has no fancy bells and whistles. It contains boxes (cells) with easy-to-read labels, and clear instructions if necessary. It also contains conditional formatting and even built-in Data Validation to verify the data.

The input screen is the inner workings of the vehicle. The final quote, the PDF you receive after clicking the button called “Create Quote,” is the vehicle’s body. And you want that body to be a sports car.

Sports car

3. Lastly, use VBA code to make it sing and dance

“VBA” stands for “Visual Basic for Applications.”

Visual Basic is a programming language built right into Excel, the “Application.” You can view this programming language in any Excel file by pressing Alt + F11 together on your keyboard.

This is an advanced subject, and it’s really what makes Excel stand out above all other similar tools out there.

You can do almost anything with Excel VBA.

For example, using VBA you could:

Store all your past quotes in a database.
Search for old quotes by client, date, cost, subject, etc., and import them into a new quote.
Automate much of the number crunching.
Have the code manipulate the final PDF’s look in tiny ways which aren’t possible with Excel alone.
Import quoting details from external software such as Sage, Salesforce, or whatever software you are using in your company.

This is really where Eppert Consulting shines. I say that with no conceit or lack of humility. I say it merely as a fact. This is our forte: Creating Excel tools that go beyond the usual, beyond the norm, using Excel VBA coupled with Excel formulas and beautiful templates, which are a piece of cake to use.

Anyone can “Record a Macro,” but to transform Excel from a mere Volkswagen Beetle into an Audi RS7, you have to know how to program.

Summary

Still, even without VBA or a Macro, the simplicity of the steps is:

Input the data in one sheet, replete with locked cells, conditional formatting, data validation and all the usual built-in Excel tools.
Suck in the data using formulas into another presentation sheet which you can then save as a PDF.

If you’re looking for more advanced features, like VBA programming, you can give us a call or send us an email using the form below so we can help you.

Tag: Excel