ReproducibiliTea Podcast

Today, Will talks to David Reinstein about scientific publishing and The Unjournal.

The Unjournal: https://www.unjournal.org

The Unjournal on LinkedIn: https://www.linkedin.com/company/unjournal/

To get the latest updates: https://bit.ly/ujupdates

To apply for positions at the Unjournal: https://bit.ly/Ujwork

Will and David’s extended notes for the episode: https://docs.google.com/document/d/13rL6mq71GD6gPv5wBHr_lfX6YSKkLK9n6EAVfGhZhCk/edit?usp=sharing

For more info go to https://ReproducibiliTea.org

For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Will sits down with Björn Jorges and Sabrina Hansmann-Roth to discuss the role of academic societies in the science reform movement.

The poster session: https://www.visionsciences.org/2023-pre-data-collection-poster-session-satellite/

Korbmacher, M., Azevedo, F., Pennington, C., Hartmann, H., Pownall, M., Schmidt, K., ... & Evans, T. (2023). The replication crisis has led to positive structural, procedural, and community changes. Communications Psychology.

https://gala.gre.ac.uk/id/eprint/42730842730_EVANS_The_replication_crisis_has_led_to_positive_structural_procedural_and_community_changes.pdf

As always:

For more info go to ReproducibiliTea.org

For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today Sarah chats to Zoltan Dienes Live from SIPS!

Zoltan's keynote: https://www.youtube.com/watch?v=BxdGXLOC1Cc

Reviewing labor: https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-021-00118-2?ref=refind

Peer Community In Registered Reports: https://rr.peercommunityin.org/

Flourishing Science Think Tank paper: https://mindrxiv.org/4zrmd

There are a few little audio problems in this weeks episode, so special shoutout to our transcript, which is here: https://docs.google.com/document/d/1zd4o40QLn0o5q1Eyp966EoDdvu5XtT6O/edit?usp=sharing&ouid=115964780222242468834&rtpof=true&sd=true

For more info go to ReproducibiliTea.org

For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Sarah is joined by Agata Bochynska and Matthew Good from the University of Oslo's Open Research Team to talk about how vital libraries are to Open Science.

Links from this episode:

QualiFAIR: https://www.uv.uio.no/ils/english/about/organization/tlvlab/qualifair/

TIER2: https://tier2-project.eu/

Carpentries: https://carpentries.org/

ReproducibiliTea at UiO: https://www.ub.uio.no/english/libraries/dsc/open-repro-research/reproducibilitea/index.html

Preprint mentioned by Agata in the last segment: https://osf.io/kcvra/

Agata Twitter: https://twitter.com/AgataBochynska

Agata mastodon: https://fediscience.org/@agata

Digital Scholarship Centre website: https://www.ub.uio.no/english/libraries/dsc/

For more info go to ReproducibiliTea.org

For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Sarah and Will discuss the invisible workload of making open science.

The paper on invisible workload: https://journal.trialanderror.org/pub/the-invisible-workload/release/1

The replication crisis has led to positive structural, procedural, and community changes: https://osf.io/r6cvx/

For more info go to ReproducibiliTea.org

For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today special guest Nora Serres talks with Sarah Sauve about Registered Report and how cool Bayesian statistics are.

Show notes:

Appetezer paper: Bayes factor design analysis: Planning for compelling evidence https://link.springer.com/article/10.3758/s13423-017-1230-y

Nora’s thesis: https://www.duo.uio.no/bitstream/handle/10852/95323/5/Master-thesis_NS.pdf

On registered reports:

https://www.cos.io/initiatives/registered-reports

https://osf.io/preprints/metaarxiv/x7aqr

https://www.nature.com/srep/journal-policies/registered-reports

PCI Registered Reports: https://rr.peercommunityin.org/PCIRegisteredReports/help/guide_for_authors

On Bayesian/statistics:

Bayesian Spectacles: https://www.bayesianspectacles.org/

Understanding Psychology as a Science by Zoltan Dienes: https://link.springer.com/book/9780230542303

For more info go to ReproducibiliTea.org

For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

The transcript is available here:

https://docs.google.com/document/d/1CCk__36WxOxuuFc1JX8lou03HvGVRci4/edit?usp=sharing&ouid=115964780222242468834&rtpof=true&sd=true

Today, Will and Jan sit down with special guest and author of the paper "A large-scale study on research code quality and execution" Ana Trisovic to talk about the reproducibility of analysis code. Also windows.

Links from this episode:

Ana's paper: https://www.nature.com/articles/s41597-022-01143-6

Gratitude package: https://github.com/Pakillo/grateful

For more info go to ReproducibiliTea.org

For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

! Important Update !

Since recording, UCU have announced that they will "ringfence £250,000 for members facing deductions for undertaking forms of ASOS, including the marking and assessment boycott." This is reassuring news for workers at universities where the VCs have threatened 50-100% pay deductions. But still no new offers from employers.

Today, Sophia and Jan are sitting down with @IlsePit(@fediscience.org) and @KatarinaWarren (k-almeidawarren.com) to chat about the ongoing UCU strikes in the UK, and how it is for ECRs to take industrial actions.

"The University and College Union (UCU) represents over 120,000 academics, lecturers, trainers, instructors, researchers, managers, administrators, computer staff, librarians, technicians, professional staff and postgraduates in universities, colleges, prisons, adult education and training organisations across the UK" (https://www.ucu.org.uk/article/1685/About-UCU)

You can support the UCU here: https://www.ucu.org.uk/

For more info go to https://ReproducibiliTea.org

For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Will, Helena and Jan talk about how we assess research and that it is weird we never really learned how to do Peer Review.

Papers we discussed:

Responsible Research Assessment Should Prioritize Theory Development and Testing Over Ticking Open Science Boxes: https://psyarxiv.com/ad74m/

A consensus-based tool for evaluating threats to the validity of empirical Research: https://psyarxiv.com/fc8v3

Will's blog post on peer review: https://williamngiam.github.io/blog/my_peer_review_process

The tool for checking p-Values: statcheck.io

The tool for checking mean values: https://journals.sagepub.com/doi/10.1177/1948550616673876

For more info go to ReproducibiliTea.org.

Do you have questions? Comments? Wanna say hi? You can reach us here: https://forms.gle/H6jgUzbbpyauLxUC8

We mentioned lots of resources this week, brace yourselves!

Books that changed our worlds

How To Read a Paper: The Basics of Evidence-Based Medicine by Trisha Greenhalgh

https://www.amazon.co.uk/How-Read-Paper-Evidence-Based-Medicine/dp/1444334360

How to Win Campaigns by Chris Rose

https://www.amazon.co.uk/How-Win-Campaigns-Steps-Success/dp/1853839620

Pollution is Colonialism by Max Liboiron

https://www.dukeupress.edu/pollution-is-colonialism

A Student’s Guide to Open Science by Charlotte Pennington

https://www.mheducation.co.uk/a-student-s-guide-to-open-science-using-the-replication-crisis-to-reform-psychology-9780335251162-emea-group

The Art of Statistics: Learning from Data by David Spiegelhalter

https://www.penguin.co.uk/books/294857/the-art-of-statistics-by-spiegelhalter-david/9780241258767

Tools we mention:

FORRT Academic Wheel of Privilege: https://www.youtube.com/watch?v=mzEdTyA06cU&ab_channel=FORRTproject

Ioannidis et al. (2019): https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000384 who provide a means to adjust research impact for self-citations using a newly developed tool

Baccini et al. (2019): https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0221212 calculate an “inwardness” metric which shows bias to researchers’ own country

Citation Diversity Statement: tools that can help authors assess the citation diversity statistics of their papers (see, for example, https://github.com/dalejn/cleanBib#instructions)

Jane Lawrence Sumner’s Gender Balance Assessment Tool (GBAT) https://jlsumner.shinyapps.io/syllabustool/

Groups/projects working on highlighting under-cited work: ABRIR (https://abrirpsy.org/), Marginalia (https://www.marginaliascience.com/) and FORRT (https://forrt.org/neurodiversity/)

Other relevant links/resources:

Evidence-Based Toxicology

https://www.tandfonline.com/journals/tebt20

Evidence-Based Toxicology Collaboration

https://www.ebtox.org/

Living a Feminist Life by Sara Ahmed

Peer review episodes of Secret Feminist Agenda: https://secretfeministagenda.com/2020/04/07/bonus-episode-season-3-peer-review-of-secret-feminist-agenda/ and https://secretfeministagenda.com/2020/04/07/bonus-episode-response-to-review-of-season-3/

Algorithms are racist & sexist: Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. nyu Press.

#MHAWS: Mirya Holman's Aggressive Winning Scholars Newsletter: https://miryaholman.substack.com/

MetaDocencia - https://www.metadocencia.org/