In this special guest feature, Kevin Dudak from Spectra Logic looks at why never-ending growth of data requires both daily diligence and a long-term outlook.
My last blog entry started the down the path of accidental Big Data after a conversation with a customer. This customer is like many organizations, they never expected that data would grow at the rate it has, or what the value of that data would be. The result is many organizations are finding themselves behind. They have made many rational decisions and purchases over the last 10 years that have hurt as much as have helped them today.
I want to focus on one area that doesn’t get as much attention as it should, the challenge of infrastructure knowledge. With the proliferation of huge data sets and different use cases for the data, the need for storage is never ending. This has caused many organizations to grow in unplanned spurts and sometimes in multiple directions. After a couple years, their environment consists of a conglomeration of different hardware and software systems that don’t always play well with others. The bigger problem is that each product is managed differently and turnover often puts different people in place.
How do we fix this? Well, first off, try to stop making it worse. We now know that data growth is only going to accelerate. While many people laughed at the idea of a Petabyte sized data center, we should all be taking an Exabyte sized one seriously. As we look at new systems, we should consider:
- How does it scale?
- Will new people be able to manage it?
- How does it integrate with my existing systems?
- Will it support emerging technologies such as Data Analytics?
We have a new understanding of data growth and potential future uses. When you do bring new technology into your shop consider the impact that technology will have on your existing systems. Hopefully this will prevent making things more complex.
The next step is to make the current environment easier to use. Identify your organizations long term targets and goals. Then look at your systems to see what helps your organization get there and what doesn’t help it to get there. Then modify or replace them. In practice, it can be hard to set strategic goals for an organization, but you should be able to identify the biggest pain points in your environment and work to repair or replace them.
Lastly, stay on top of it all! Doing all the work isn’t going to help if you don’t stay on top of it. Much like keeping an organized garage, it requires constant maintenance. Every change, upgrade and addition to the environment moving forward needs to be thought about in the long term. Keeping an eye on the long term challenges will ultimately help with issues today and tomorrow.
