The Clearly Podcast

Send a text

The podcast discusses the issue of data lakes turning into data swamps due to the uncontrolled dumping of data. While storage is cheap, this can lead to a mess with duplicate and poor-quality data that’s hard to use. Organizations must implement processes to control what data goes into the lake, document where it’s stored, and build a data dictionary. Cleaning up existing data involves removing duplicates and poor-quality data, and appointing a data steward to oversee the process.

There's a need for a cultural shift to focus on data quality rather than quantity. Governance policies, such as data retention and clear ownership, are crucial. Proper data management can be costly but prevents inefficiency and poor decision-making. The importance of having the right foundations, including people, processes, and technology, along with business buy-in, is emphasized.

In conclusion, maintaining a healthy data lake requires well-defined processes, governance, and a culture that values data quality. Next week's topic will cover how to migrate to the cloud, specifically from SQL on-prem to SQL in the cloud.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

The podcast discusses Fabric’s recent move to General Availability (GA) following its announcement at Microsoft Ignite. GA status means Fabric is now a fully released product, ensuring stability and predictability for production use.

The discussion then moves to pricing, detailing the different SKUs ranging from F2 to F2048. Prices vary by region, with significant savings available for those who choose to reserve capacity. Power BI licensing is included starting at F64, making it a cost-effective option for organizations that require both Fabric and Power BI.

Storage costs for Fabric are additional but competitively priced at around £20 per terabyte per month.

The podcast highlights that Fabric is suitable for both SMBs and larger enterprises, with lower SKUs fitting well within the SMB market. The flexibility in capacity units across different workloads is emphasized.

For those starting with Fabric, it’s advised to assess data sizes, sources, and movement frequency, using existing Azure calculators for sizing. When considering a proof of concept (POC), it’s recommended to start with a small instance and choose a dataset that showcases Fabric’s capabilities beyond what Power BI alone can do.

The podcast concludes by noting that existing Power BI premium capacities now support all Fabric workloads, making it a valuable addition for current Power BI users. The hosts express excitement for the next week’s surprise topic and thank their listeners.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

1. Avoid Controversial Topics:
   • Steer clear of discussing politics, religion, finance, and sports to prevent conflicts with customers.
2. Set Boundaries:
   • Maintain professional relationships with clients. Avoid romantic entanglements and excessive socializing, such as getting drunk with them.
3. Avoid Criticizing Customer Staff:
   • Stay out of office gossip and politics. Remember, you're an external consultant, not part of the client's internal team.
4. Avoid Swearing:
   • Be polite and stay calm. While some environments might tolerate mild swearing, it's best to avoid it, especially racial or gender slurs.
5. Observe Dress Codes and Office Norms:
   • Dress appropriately, especially on your first visit. Adjust to the client's office culture once you're familiar with it.
6. Secure Customer Data:
   • Follow security policies, use strong passwords, and ensure data encryption. Protect both digital and printed data.
7. Be Accurate:
   • If unsure about something, say you'll find out and get back to them. It's better to verify information than to provide incorrect answers.
8. Don't Promise What You Can't Deliver:
   • Be honest about capabilities and deadlines. Avoid overpromising to secure a sale or project.
9. Don't Be Arrogant:
   • Confidence is good, but arrogance can be off-putting. Be aware of your client's expertise and respect their knowledge.
10. Respond When You Promise To:
    • Keep your commitments and follow through on responses. If you need more time, communicate that clearly.

Next week’s topic will be updates on Fabric.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

Today's discussion focused on how customers should resource projects with their own people. Often, projects are under-resourced with the expectation that consultants will do all the work, which is not effective for long-term success. Clear communication and proper resource allocation are essential.

Clients frequently struggle to balance day-to-day tasks with project commitments. It's crucial for management to empower their teams to prioritize project work. People involved in the project should be given the time and authority to make necessary decisions.

Some individuals are naturally better at balancing multiple tasks, but this can be a learned skill through experience. Training and team-building are important, especially if the team hasn't worked together before.

When issues arise in a client project, it's important to address them quickly, ideally through the project manager. Clear communication from the start, including defining roles and responsibilities, can prevent confusion and delays. Setting expectations upfront about client involvement can help ensure smoother project execution.

Testing is a critical part of the process but is often neglected. Clients need to actively participate in testing to ensure the product meets their requirements. Documentation, although tedious, is also essential for project success.

To summarize, clients need to assign the right people, free up their time, empower them, communicate clearly, and actively participate in testing. Training and setting expectations are crucial for effective project resourcing.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

In this podcast episode, the hosts discuss the complexities of client requirements gathering. They emphasize that understanding and gathering requirements isn't as straightforward as it seems and requires clear communication and documentation. Key points include:

• Importance of Clear Requirements: Gathering and understanding requirements correctly is crucial to avoid misunderstandings and project issues. Writing down and replaying requirements to confirm understanding is essential.
• Role Responsibilities: Clients should have a dedicated project manager to manage and own the project. Consultants, particularly business analysts, should document and replay the requirements to ensure clarity.
• Handling Changes: Requirements often change during a project. It's important to document and formally agree on changes to manage scope and expectations effectively.
• Effective Gathering Techniques: Workshops, whiteboard sessions, and prototyping help visualize the end result and clarify requirements. Understanding current processes and definitions across departments is essential for consistency.
• Common Pitfalls: Issues arise when no one is empowered to make final decisions, leading to confusion and delays. Miscommunication within the client’s organization and lack of replaying requirements can also cause problems.
• Advice for Clients: Empower individuals to make decisions, ensure consensus on definitions, allocate time for the process, and work closely with consultants. Consultants should ask questions and guide the client through the process.

The discussion concludes by reaffirming that agile methodologies are still valuable, emphasizing the need for well-defined requirements in each phase of the project. The hosts plan to discuss time management in projects in the next episode.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

In this podcast, the team discusses common data horror stories in IT consulting. They highlight the issue of hiding bad data with filters in reports instead of fixing errors at the source, leading to bigger problems during system migrations. Another problem discussed is trying to use tools like Power BI to mimic Excel functionalities, which leads to alignment and maintenance issues since Power BI and similar tools are not designed to replace Excel.

The conversation touches on the issues with manipulating data in Excel before importing it into Power BI, causing inconsistent data and errors. There's also a focus on the importance of cleaning data before migration to avoid transferring existing problems to new systems.

They mention the challenges of treating estimates as fixed prices and the tendency of clients to ask for continuous changes, causing scope management issues. Another point raised is the problematic practice of turning prototypes into production reports, leading to more issues because prototypes are not meant for long-term use.

The discussion also covers the problem of having poorly defined measures across different departments, resulting in inconsistencies in reporting. Lastly, they emphasize the importance of understanding one's own data to ensure accurate and effective data management and reporting. The episode concludes with a reminder to stay safe and avoid nightmares from data horrors.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

Today's discussion focused on the ethics of AI and managing customer data. There is a growing concern about how AI models, both internal and external, handle sensitive information. The conversation aimed to clarify what actions are acceptable, the conditions for those actions, and what should be avoided. It emphasized the importance of understanding the ethical implications of uploading sensitive data to AI models like ChatGPT.

Key points included the need for transparency about the training data used in AI models to prevent biases. Public models often lack transparency, making it difficult to ensure unbiased outputs. Legal considerations, such as GDPR, require obtaining permission for processing personal data and maintaining transparency with customers.

There is a distinction between using public models like ChatGPT and private internal models. While responsibilities remain similar, the availability of information differs. With public models, the lack of transparency might necessitate avoiding them or conducting thorough testing to ensure unbiased results. Internal models provide more control over the training data but require careful consideration of data quality.

Data leaving the organization, even for software development, requires strict control and understanding of data security measures. Contracts should specify data usage and security, ensuring compliance with data protection laws. The conversation also touched on the differences between generative AI and RPA in processing data, emphasizing the need for human oversight in AI processes.

There was a discussion about the potential move towards niche AI models specialized in specific fields, which could offer better results and reliability. Specialized models in areas like medicine can focus on quality inputs and provide accurate results, whereas general models create broad excitement but might lack depth.

The conversation highlighted the importance of caution when uploading data to public AI models, considering legal, commercial, and reputational risks. Organizations should start with anonymized test data and understand their internal data processes before leveraging AI. Specialist AI models might be more beneficial than general ones.

Final thoughts emphasized reading the terms and conditions of any AI tool and the need for AI providers to adopt and follow codes of practice for safety and ethics. Organizations should try AI cautiously, ensuring data security and privacy. The future likely holds more niche AI models that will be useful and beneficial.

Next week's topic will be the importance of requirements gathering for BI and reporting, with Shailan preparing the briefing notes.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

Today's discussion revolves around whether ChatGPT could potentially take over jobs, particularly in the field of BI and Power BI modeling. ChatGPT is a generative AI that creates text-based outputs by analyzing large amounts of text data from the internet. While it can generate code, it might not always function correctly and can produce errors if not carefully reviewed.

In practice, some clients use ChatGPT for basic tasks, like creating flow diagrams for Power Platform apps. While useful, these outputs often require further refinement. It's essential to be cautious with data privacy when using ChatGPT, as sensitive information should not be uploaded.

Despite its limitations, ChatGPT can be a valuable tool if used correctly. It can handle standard tasks and free up time for more complex work, allowing professionals to focus on strategic tasks and client engagement. The tool requires practice and a clear understanding of its strengths and weaknesses to be used effectively.

It's also important to note that using ChatGPT leaves a data footprint, and the latest version, ChatGPT-4, is a paid service. The conversation concludes with an agreement to discuss AI ethics and data handling in the next session.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

Microsoft Fabric is a new data analytics platform from Microsoft that consolidates various Azure tools, including Synapse Analytics, Azure Data Factory, and real-time data streaming, under a single architecture called OneLake. OneLake allows these tools to operate on the same underlying platform, simplifying collaboration across teams and enhancing productivity.

The consolidation is significant because it eliminates data silos, making it easier to integrate real-time analytics, data warehousing, and data science. For customers currently using these technologies, Microsoft Fabric offers a cohesive, single-vendor solution with tools like serverless compute, Power BI, and data factory. This integration simplifies cost management as everything is under one subscription, reducing the complexity of multi-vendor setups.

However, while Fabric's ease of use is advantageous, it also necessitates proper planning and oversight to avoid inefficient and costly implementations. Poorly constructed solutions can severely impact performance, highlighting the need for trained staff to manage these tools.

Vendor lock-in is a potential concern, but Fabric's capability to integrate with external data sources like Amazon S3 helps mitigate this risk. Organizations often choose a primary platform like Microsoft or AWS based on strategic needs, and despite potential lock-in, the benefits of a cohesive and integrated platform are substantial.

Fabric uses a consumption-based pricing model, so it’s crucial for organizations to manage and monitor their usage closely. Budget caps and Azure calculators can help predict and control costs effectively.

The initial target audience for Microsoft Fabric includes large enterprises and existing Synapse Analytics users. Over time, mid-sized organizations using various Azure tools might also find value in consolidating under Fabric. Overall, Microsoft Fabric offers a promising solution for various data needs, reducing costs, and improving collaboration through its OneLake architecture. Proper planning and usage are essential to unlocking its full potential.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Send a text

The discussion centers on the temptation to import all data into a system during a data project, highlighting several key issues with this approach. Importing all data can create clutter, making it difficult to find relevant information, especially in systems like Dynamics 365 F&O that have thousands of tables. This complexity can lead to inefficiencies and challenges in data retrieval.

Using all available tables can slow down data refresh processes, particularly when clients require frequent updates. Incremental refreshes, while somewhat mitigating this, still involve complex calculations. Overloading users with too much data can overwhelm them, making it harder for them to create their own reports effectively.

Field naming conventions can add to the confusion, especially when fields are coded or not clearly named, making it challenging for untrained users to navigate. Additionally, error handling becomes more difficult with more data, as spotting and fixing errors in numerous fields is more time-consuming.

There's also an environmental consideration; storing and processing unnecessary data consumes significant resources, contributing to environmental concerns. Future data migrations become complicated when unnecessary data is stored, making the process more time-consuming and costly.

The top reasons to avoid importing all data include ensuring data relevance and focus, reducing implementation time and costs, managing processing costs, enhancing security, simplifying error handling, and mitigating environmental impact. The recommendation is to start with the end in mind, identifying necessary reports and working backward to determine essential data, thereby finding a balance and avoiding extremes in data importation.

You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/