Showing posts with label Hadoop Institute. Show all posts
Showing posts with label Hadoop Institute. Show all posts

Sunday, 12 November 2017

Benefits of Big Data Processing

Ability to process 'Big Data' brings in multiple benefits, such as-
• Businesses can utilize outside intelligence while taking decisions
Access to social data from search engines and sites like facebook, twitter are enabling organizations to fine tune their business strategies.

• Improved customer service
Traditional customer feedback systems are getting replaced by new systems designed with 'Big Data' technologies. In these new systems, Big Data and natural language processing technologies are being used to read and evaluate consumer responses.

• Early identification of risk to the product/services, if any
 
• Better operational efficiency
'Big Data' technologies can be used for creating staging area or landing zone for new data before identifying what data should be moved to the data warehouse. In addition, such integration of 'Big Data' technologies and data warehouse helps organization to offload infrequently accessed data.

Saturday, 5 August 2017

DevOps Automation for Faster and Continuous Product Release

DevOps

Although the above example is somewhat crude it is a fair assessment of what application development can be like end-to-end. Everyone in the industry knows that this is the 'normal' state of affairs and accept that it is less than perfect. DevOps has begun to appear on the scene as the answer to the traditional silo approach. DevOps attempts to remove the silos and replace them with a collaborative and inclusive activity that is the Project. Application Development and Solution Design benefit from DevOps principles.

What needs to be done to remove silos:

    Change the working culture
    Remove the walls between teams (and you remove the silos)

Keys:

    Communication, Collaboration, Integration and Information Sharing

Easy to say and hard to do.

Most SMEs like to keep their information to themselves. Not true of all but, of many. It's part of the traditional culture that has developed over many years. Working practices have made change difficult. Management of change is one of the most challenging tasks any company can embark on. Resistance will be resilient as it is important that people give up something to gain something. Making it clear what the gains are is imperative. People will change their attitudes and behaviours but, you have to give them really good reasons to do so. I've found that running multi-discipline workshops for the SMEs has proven an effective method of encouraging information-sharing and the breaking down of those 'pit-walls'.

Explaining to the teams what DevOps is and what it is supposed to achieve is the first part of the educational process. The second is what needs to be done.

State specific, measurable objectives:

    Implement an organization structure that is 'flat'. If we espouse horizontal scaling, why not horizontal organizations?
    Each App-Dev or Solution-Dev is a project and the team is end-to-end across the disciplines
    Implement ongoing informational exchange and reviews
    Make sure that everyone signs up to DevOps and understands the paradigm

What is DevOps

Just like the Cloud paradigm it is simply another way of doing something. Like Cloud it has different definitions depending on to whom you are speaking at the time.

Wikipedia states: Because DevOps is a cultural shift and collaboration between development and operations, there is no single DevOps tool, rather a set or "toolchain" consisting of multiple tools. Generally, DevOps tools fit into one or more categories, which is reflective of the software development and delivery process.

I don't think that this is all DevOps is. The inference is that DevOps is concerned only with application development and operations. I do not believe that. I believe that DevOps is a paradigm and that like other IT 'standards' and paradigms it is relevant to all IT and not just applications. By removing the partitions between each practice in the chain and having all the key players involved from day one, as part of an inclusive and collaborative team, the cycle of application development and solution design becomes a continuous process that doesn't have to divert to consult each required expert. No-one needs to throw a document over the wall to the next crew. Each document is written within the collaboration process and this has to make the document more relevant and powerful. Imagine that the project team is always in the same room from concept to deployment and each expert is always available to comment on and add to each step of that project. How much better than the traditional method where it can take days to get an answer to a simple question, or to even find the right person to ask.

The mantra is: Develop, Test, Deploy, Monitor, Feedback and so on. This sounds application-orientated. In fact, it can apply to the development of any IT solution. Like ITIL, TOGAF and the Seven Layer Reference Model it can be applied to any and all IT activities from development right through to support services. DevOps puts us all on the same page from the start to the finish.