#topic/josh1-1 what do you think about these distinctions | **Category** | **Definition** | **Inputs** | **Outputs** | **Tools Used** | | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | **Data Engineering** | Consuming, manipulating, storing data (not data collection)<br><ul><li>managing data pipelines</li><li>cleaning data for analysis</li></ul> | <ul><li>API</li> <li>databases</li> <li>files</li></ul> | Structured, clean data ready for analysis | **ETL Tools**: SSIS, Automate <br>**Data Storage**: SQL Server <br>**APIs & Web Scraping**: Python, BeautifulSoup | | **Analysis** | <li>querying data</li><li>exploratory investigation</li><li>building predictive models</li> | Structured data (from data engineering) | <ul><li>Insights</li> <li>predictions</li><li>models</li></ul> | **Querying**: SQL, DAX<br>**Exploratory Tools**: Jupyter, Python, Excel <br>**Modeling**: Python, Scikit, AutoML | | **Reporting** | The presentation of insights for stakeholders and decision-makers. | Processed data (from data engineering) | <ul><li>Visual reports</li> <li>summaries</li> <li>dashboards</li></ul> | **BI Tools**: Power BI <br>**Report Generation**: Webi, PDF, Excel <br>**Distribution**: Email, Web portals, publications | | **Database Administration** | Managing and optimizing database systems<br><ul><li>performance tuning</li><li>security</li><li>maintenance</li><li>backup/recovery</li></ul> | <ul><li>Database systems</li><li>Monitoring data</li><li>User requirements</li></ul> | <ul><li>Optimized databases</li><li>Security protocols</li><li>Backup systems</li></ul> | **Management Tools**: SSMS<br>**Monitoring**: redgate<br>**Backup**: SQL Server Backup<br>**Security**: Active Directory | | **Applications** | Development and maintenance of data-driven applications<br><ul><li>CRUD interfaces</li><li>business process automation</li><li>data entry systems</li></ul> | <ul><li>Business requirements</li><li>User feedback</li><li>Data models</li></ul> | <ul><li>Web applications</li><li>Internal tools</li><li>Data entry systems</li></ul> | Automate, Power Apps<br>**Backend**: C#, Python<br>**Frameworks**: .NET<br>**UI**: Angular |