news | B101nfo

experDesign: follow up

Sun, 09 Apr 2023 00:00:00 +0000

I am happy to announce a new release of experDesign. Install it from CRAN with:

install.packages("experDesign")
library("experDesign")

This new release has focused in more tricky aspects when designing an experiment:

Checking the samples of your experiment.
How to continue stratifying your conditions after some initial batch.

These functions should be used before carrying out anything once you have your samples collected. You can use these functions and make an informed decision of what might happen with your experiment.

Checking your samples

The new function check_data() will warn you if it finds some known issues with your data.

library("experDesign")
library("MASS")

If we take the survey dataset from the MASS package we can see that it has some issues:

data(survey, package = "MASS")
check_data(survey)
## Warning: Two categorical variables don't have all combinations.
## Warning: Some values are missing.
## Warning: There is a combination of categories with no replicates; i.e. just one
## sample.
## [1] FALSE

While if we fabricate our own dataset we might realize we have a problem

rdata <- expand.grid(sex = c("M", "F"), class = c("lower", "median", "high"))
stopifnot("Same samples/rows as combinations of classes" = nrow(rdata) == 2*3)
check_data(rdata)
## Warning: There is a combination of categories with no replicates; i.e. just one
## sample.
## [1] FALSE
# We create some new samples with the same conditions
rdata2 <- rbind(rdata, rdata)
check_data(rdata2)
## [1] TRUE

One might decide to go ahead with what is available or use only some of those samples or wait to collect more samples for the experiment

Follow up

Imagine you have 100 samples that you distribute in 4 batches of 25 samples each. Later, you collect 80 more samples to analyze. You want these new samples to be analyzed together with those previous 100 samples. Will it be possible? How should you distribute your new samples in groups of 25?

Using the same dataset from MASS imagine if we first collected 118 observations and later 119 more:

survey1 <- survey[1:118, ]
survey2 <- survey[119:nrow(survey), ]
# Using low number of iterations to speed the process 
# you should even use higher number than the default
fu <- follow_up(survey1, survey2, size_subset = 50, iterations = 10)
## Warning: There are some problems with the data.
## Warning: There are some problems with the new samples and the batches.
## Warning: There are some problems with the new data.
## Warning: There are some problems with the old data.

Following the previous new function it reports if there are problems with the observations. One can check each collection with check_data to know more about the problems found.

If you have already performed the experiment on your observations you can also check the distribution:

# Create the first batch
variables <- c("Sex", "Smoke", "Age")
survey1 <- survey1[, variables]
index1 <- design(survey1, size_subset = 50, iterations = 10)
## Warning: There might be some problems with the data use check_data().
r_survey <- inspect(index1, survey1)
# Create the second batch with "new" students
survey2 <- survey2[, variables]
survey2$batch <- NA
# Prepare the follow up
all_classroom <- rbind(r_survey, survey2)
fu2 <- follow_up2(all_classroom, size_subset = 50, iterations = 10)
## Warning: There are some problems with the data.
## Warning: There are some problems with the new samples and the batches.
## Warning: There are some problems with the new data.
## Warning: There are some problems with the old data.
tail(fu2)
## [1] "NewSubset2" "NewSubset2" "NewSubset2" "NewSubset2" "NewSubset2"
## [6] "NewSubset3"

Using this function will help to decide which new observations go to which new batches.

Closing remarks

The famous quote from Fisher goes:

“To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.”

This emphasizes the importance of involving a statistician early on in the experimental design process.
Unfortunately, in some cases, it may be too late to involve a statistician in the experimental design process or the reality of unforeseen circumstances messed the design of your carefully planned experiment.

My aim with this package is to provide practical tools for statisticians, bioinformaticians, and anyone who works with data. These tools are designed to be easy to use and can be used to analyze data in a variety of contexts. Let me know if it is helpful in your case.

Reproducibility

## ─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────
##  setting  value
##  version  R version 4.2.2 (2022-10-31)
##  os       Ubuntu 22.04.2 LTS
##  system   x86_64, linux-gnu
##  ui       X11
##  language en_US
##  collate  en_US.UTF-8
##  ctype    en_US.UTF-8
##  tz       Europe/Madrid
##  date     2023-04-09
##  pandoc   2.19.2 @ /usr/lib/rstudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
## 
## ─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────
##  package     * version  date (UTC) lib source
##  blogdown      1.16     2022-12-13 [1] CRAN (R 4.2.2)
##  bookdown      0.33     2023-03-06 [1] CRAN (R 4.2.2)
##  bslib         0.4.2    2022-12-16 [1] CRAN (R 4.2.2)
##  cachem        1.0.7    2023-02-24 [1] CRAN (R 4.2.2)
##  cli           3.6.1    2023-03-23 [1] CRAN (R 4.2.2)
##  digest        0.6.31   2022-12-11 [1] CRAN (R 4.2.2)
##  evaluate      0.20     2023-01-17 [1] CRAN (R 4.2.2)
##  experDesign * 0.2.0    2023-04-05 [1] CRAN (R 4.2.2)
##  fastmap       1.1.1    2023-02-24 [1] CRAN (R 4.2.2)
##  htmltools     0.5.4    2022-12-07 [1] CRAN (R 4.2.2)
##  jquerylib     0.1.4    2021-04-26 [1] CRAN (R 4.2.2)
##  jsonlite      1.8.4    2022-12-06 [1] CRAN (R 4.2.2)
##  knitr         1.42     2023-01-25 [1] CRAN (R 4.2.2)
##  MASS        * 7.3-58.1 2022-08-03 [2] CRAN (R 4.2.2)
##  R6            2.5.1    2021-08-19 [1] CRAN (R 4.2.2)
##  rlang         1.1.0    2023-03-14 [1] CRAN (R 4.2.2)
##  rmarkdown     2.20     2023-01-19 [1] CRAN (R 4.2.2)
##  rstudioapi    0.14     2022-08-22 [1] CRAN (R 4.2.2)
##  sass          0.4.5    2023-01-24 [1] CRAN (R 4.2.2)
##  sessioninfo   1.2.2    2021-12-06 [1] CRAN (R 4.2.2)
##  xfun          0.37     2023-01-31 [1] CRAN (R 4.2.2)
##  yaml          2.3.7    2023-01-23 [1] CRAN (R 4.2.2)
## 
##  [1] /home/lluis/bin/R/4.2.2
##  [2] /opt/R/4.2.2/lib/R/library
## 
## ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Writing a thesis with bookdown

Mon, 09 May 2022 00:00:00 +0000

On this post I am documenting the experiences I had writing my PhD thesis with bookdown. I made the thesis in web and pdf format (and epub) to make more available the thesis. Most of the experiences and advise I’ll share here are based on my experiences to improve the pdf format. It is the most important format as ultimately is what I’m going to use for printing.

First of all you should know there is a package thesisdown with a few templates for some universities. If yours is there, or if you want to learn how are they you can have a look at the files.
On my case I didn’t have any template and the university guidelines are not long, (have two compulsory pages at the beginning and have 5 sections). That’s why I tweaked the default format inspired by the recent thesis defended on my group.

Well, without further delay let’s dive in things I learned:

Important

Captions

knitr::kable places the captions on tables at the top (by design, see issue #1189), while knitr places the captions on the bottom of figures. So if you want to have all the captions below the element you’ll need to use a different package for it (booktable, or others).

If you want to have short captions for an easy readable table of figures and table of tables you’ll need to use kable(short.caption = "TOC", caption = "Long caption below the table").

In addition, on kable if you use something like Häsler you’ll need to convert this “ä” to “\u00E4”.

I also wanted to highlight and differentiate the captions. I ended up using the caption package:

\usepackage{caption}
% Set in bold the numbering of tables and chapters
\captionsetup{labelfont=bf,width=\textwidth}

The \textwidh is to make more with the captions otherwise they just spans the size of the table or figure.

Repeating text

If you find yourself repeating some text to explain some figures, legends or tables you can use text references.

(ref:foo) Define a text reference **here**.

Then you can use (ref:foo) to repeat the same text.

Although formatting cannot be applied afterwards (i.e. **(ref:foo)**) it is handy to just write once and avoid repetition (And also if to keep backwards compatibility you can’t use the new special comment #| to specify chunk options).

Placing options

Many latex instructions go to the index.Rmd file.

The once I included are:

split_by: chapter
link-citations: true 
always_allow_html: true
colorlinks: yes
# https://bookdown.org/yihui/rmarkdown-cookbook/latex-variables.html
# links-as-notes: true # Only activate for actual printing
fontfamily: libertine
fontsize: 12pt
papersize: a4 # The printed size of the thesis
acronyms:
  loa_title: ""
  insert_loa: false
  sorting: usage
  include_unused: false
  fromfile: ./style/acronyms.yml
geometry:
 - top=25.4mm
 - bottom=25.4mm
 - left=25.4mm
 - right=25.4mm
 - bindingoffset=6.4mm
 - asymmetric
classoption: 
  - twoside
  - openright
lot: yes
lof: yes

split_by in the html format how to move to the next section.
link-citations Add a link to the citation?
colorlinks If links should have a color
links-as-notes Instead of having hyperlinks have them included as notes. It is useful for printing where the reader doesn’t have the option to click a link but might be interested in knowing more.
fontfamily and fontsize decide which font and size will be used.
papersize this chooses the available space and greatly affects the position of figures and tables, which can float on the text according to LaTeX algorithm.
acronyms Configuration of the acronyms
geometry Defines the margins, consider that on books the central zone will not be readable. The bindingoffset adds some space to make it easier reading.
classoption Options for the book format
lot and lof indicate if list of tables (lot) and list of figures (lof) should be included on the pdf output.

The book_filename if present on index.Rmd is overwritten by what is on _bookdown.yml but be careful also on what goes to _bookdown.yml and on the specific format on _output.yml .

Nice little tricks

Dedication

Looking at the source code of the bookdown book I found that the correct way was to use before_body option.

Chapter thumb

One thing I liked from other thesis is the ability to have on the printed edition a little mark on the side of the page to find a section.

My first search showed that it was possible, but I didn’t want to load the tikz package. I ended up using this solution after adding and modifying the colors, changing the size and position.

\usepackage[scale=1,angle=0,opacity=1,contents={}]{background}
\usetikzlibrary{calc}
\usepackage{ifthen}
\usepackage{lipsum}
% auxiliary counter
\newcounter{chapshift}
% the list of colors to be used (add more if needed)
\newcommand\BoxColor{%
  \ifcase\thechapshift blue!30\or red!30\or olive!30\or magenta!30\or teal!30\or lime!30\or orange!30\or violet!30\or brown!30\else yellow!30\fi}
% the main command; the mandatory argument sets the color of the vertical box
\newcommand\ChapFrame{%
  \def\TitleText{\leftmark}%
  \AddEverypageHook{%
    \ifthenelse{\isodd{\value{page}}}
      {\backgroundsetup{
        contents={%
          \begin{tikzpicture}[overlay,remember picture]
          \node[fill=\BoxColor,inner sep=0pt,rectangle,text width=1cm,
            text height=3cm,align=center,anchor=north east]
          at ($ (current page.north east) + (-0cm,- 3*\thechapshift cm) $)
          % {\rotatebox{90}{\parbox{4cm}{%
          %   \centering\textcolor{black}{\scshape\thechapshift}}}};
          {};
          \end{tikzpicture}
        }%
      }
    }
    {\backgroundsetup{
      contents={%
        \begin{tikzpicture}[overlay,remember picture]
        \node[fill=\BoxColor,inner sep=0pt,rectangle,text width=1cm,
          text height=3cm,align=center,anchor=north west]
        at ($ (current page.north west) + (-0cm,-3*\thechapshift cm) $)
        % {\rotatebox{90}{\parbox{4cm}{%
        %   \centering\textcolor{black}{\scshape\thechapshift}}}};
        {};
        \end{tikzpicture}
      }
    }
  }
\BgMaterial}%
\stepcounter{chapshift}
}

This code basically means that I need to add \ChapFrame when I want the chapter thumb (I didn’t know the name before searching this). Once started it changes colors according to \chapshift which is automatically incremented by \ChapFrame. I also set to change position according to \thechapshift so that they make a stair.

The code basically changes the position of the mark if the page is even or odd, so that it is always on the outer side of the booklet. The size is width=1cm, height=3cm with text inside it. If you want text I recommend either short titles or the chapter number \chapter to ensure it is readable.

A tiny trick I learned was to reset the counter with \afterpage{\setcounter{chapshift}{0}} after the bibliography so that the appendix would use the same mark from the beginning. If you want different colors for the appendix you could just create a new counter and a new \BoxColor

Index with index on it

I wanted to have the table of contents to show were it began, simply because with all the added page on the front and white pages it might be hard to find it. It is also handy when using the outline of the pdf version to go back the the index to then move to another section.

Simply loading the \tocbibind packages was enough:

\usepackage{tocbibind}

Screenshot of the outline and the real index. The outline has the same content as the real index.

Note: I found a “bug” were the appendix link goes to the bibliography (the previous chapter) instead of the correctly displayed page.

Table of figures and tables

It was not required but I wanted a table of tables and a table of figures, to make it easier go to results of the thesis. To add them I used the tocbibind package that automatically adds it (and the options on index.Rmd wasn’t sure from the answer I found online).

\usepackage{tocbibind}

Screenshot of the first lines of the list of figures, with a short caption for each figure

Acronyms

I repeat many acronyms on the thesis and I wanted to have a brief table with them. I used the package acronymsdown which is simple, easy and works well for web and pdf.
My only wish is that it had a way to go back to were the reader was.

To place the acronyms were I wanted I had to remove the automatic title and use the following:

# Glossary {-}

\printacronyms

I add them to the beginning after the summaries of the thesis and the preface, right before the body of the thesis.

Placing floats

I included many figures and tables which makes it hard to have all of them near where they are added on the text. While it can be forced, I didn’t want that but neither I wanted them too far away.

To avoid them going after the subsection I added this command before the title of the next subsection:

\FloatBarrier

Probably it could be done automatically renewing the subsection title format, but as I only had to do this 5 times is manageable.

Following an answer, the figure floating algorithm was set with these preferences:

\usepackage{float}
\usepackage{colortbl}
\let\origfigure\figure
\let\endorigfigure\endfigure
\renewenvironment{figure}[1][2] {
    \expandafter\origfigure\expandafter[!htbp]
} {
    \endorigfigure
}

A figure and a table on the same page, the label is in bold and the text in italics, on the right side (the page is odd) the blue chapter thumb

Github actions

To render I initially used my r-lib/actions but without using any package structure. However, once I set a DESCRIPTION file with all the package dependencies it was much faster, as I could use the setup-r-dependencies action.

Probably there is also a faster way directly installing binaries with the help of bspm or the system package manager, but this was convenient enough.

Also the action to install tinytex made my job for the pdf to render much faster (from 15 minutes to 5 minutes).

Screenshot of the github actions (on push) were bookdown-web takes ~3 minutes, bookdown-epub ~2 minutes and bookdown-pdf ~5 minutes.

Empty pages

The chapters are on the right side of the book so they must end or have a blank page before. To have a completely blank page I found a simple solution, simply load a latex package emptypage.

\usepackage{emptypage}

Title pages

Where to place titles, format it with titlesec:

\usepackage{titlesec}
\titleformat{\chapter}[display]{\fontsize{32pt}{48pt}\bfseries\sffamily\filcenter}{
    \fontsize{72pt}{72pt} \thechapter \ChapFrame
} % Content on the chapter title page
{20pt}{\lsstyle}[\thispagestyle{empty} \cleardoublepage]% https://tex.stackexchange.com/a/347162/178206
\titleformat{name=\chapter, numberless}{\normalfont\huge\bfseries\filcenter}{}{20pt}{\Huge}
\titlespacing*{\chapter}{0pt}{100pt}{40pt}

The first line load the package. Then we set the format of the chapters option display and the format of text and what appears on that page. \thechapter is the title of the chapter (while \chatper is the number). \chapFrame is the new command defined to set chapter thumb.

The other benefit this had was that the title page had no page number.

A screenshot of the introduction title page. There is a number 1 and Introduction a couple of lines below. On the right side a red box 3 cm below the top.

Page numbers

I decided to use different style for page numbers

\usepackage[automark,headsepline]{scrlayer-scrpage}% sets page style scrheadings automatically
\clearpairofpagestyles
\ihead{\leftmark}
\ohead*{\pagemark}
\setkomafont{pagenumber}{}% default is \normalfont\normalcolor

Include pdfs

As part of the appendix I added my publications on their pdf format. To do so I used the following code modified from the original answer:

\usepackage{pdfpages} 
\includepdf[pages=-, pagecommand={}, templatesize={\textwidth}{\textheight  - 25pt}, trim=0 0 0 20pt,]{pdf/paper.pdf}

I had to tweak the size were it was placed to keep the page numbers and running titles. The last curly brackets indicate the location of the pdf to include.

The benefit of this is that these pages now are also numbered with the thesis style and the title.

Screenshot of the pdf included showing the page number of the thesis and the content of the article “Multi-omic modelling of inflammatory bowel disease with regularized canonical correlation analysis”

Running titles

If you have a long title such as you are probably interested on having a shorter version for it on the thesis pages.

## experDesign: stratifying samples into batches with minimal bias

\sectionmark{experDesign: paper}

I don’t think it actually made a big difference but it might we important for chapters.

Merging pdfs

As I said I needed some pages at the beginning of the thesis. But I wanted to keep the outline of the pdf, and I found an solution online explaining how to do it.

To add multiple pdf I did this:

gs -q -SDEVICE=pdfwrite -DPDFSETTINGS=/prepress -o merged.pdf page1.pdf page2.pdf thesis.pdf

Reducing pdf size

At the end the pdf was bigger than what I could send over email.

I found this answer that helped me to reduce the size and send it.

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/ebook -dNOPAUSE -dQUIET -dBATCH -sOutputFile=output.pdf input.pdf

Conclusion

The process of writing the thesis is usually one of the last steps on a PhD. I recommend to write something and avoid having a blank page. But once written you must take care of the presentation and style, this is a complete different skill than writing or research, so it can be specially exhausting.

Bookdown through the preamble and body options is great for setting your style. But if you are short of time, are tired you might benefit from working and using some of these already created solutions and just modify what you need (as I did).

To finish, so that you can see the final format it is here. There you can download it in pdf too to see most of these commands in action.

If you are writing your thesis, enjoy, keep calm and reuse other solutions!

Covid-19

Sun, 15 Mar 2020 00:00:00 +0000

Introduction

I have seen several analysis and simulations of the spread. In this entry I want to use the number of beds in hospitals, the disease duration, and the infection rate to find when will the healthcare system collapse in Spain. I’ll try to link to the resoures of the assuptions I take, or where I find the numeric values I use.

Important numbers

The reproductive number is the number of people that get the virus from a person with the virus. This is highly variable and is influenced by government decisons (lock down, travel bans, …) and social behavior (See slide 11 from this deck). Many dataset are linked in this thread of Civio.

Before any major measure in Spain was ~2.6 Source
After major measures ? (too soon to say, we’ll need to wait a week)

In order to know how many time are people ill and how much time they might need to spend at the hospital we need to know the disease duration:

~20 days until death (healthy <60 years woman) Source
37 days and still testing positive of SARS-CoV-2 Source
Person released without symptoms and after a month tested again positive Source

The saturation of the hospitals can be measured by the capacity they have. Ventilators, oxygen, doctors those numbers affect at the peace the doctors can attend patients, generally the number of beds is used:

315/100000 habitants Source

The Covid severity, from a widely circulated and verifyed source:

20% of infected need hospitalization (mild cases)
5% of infected are critically ill (severe cases)

Also we can check the difference in symptoms on the population at South Corea and Italy, via this tweet that lead to this article. Also the case of Iceland

Some of them will not recover despite any treatments by the doctors, this is the letality rate. I’ve taken the numbers from a webminar of the lead epidemiologist of Hospital Clínic which was done the 5th of March.

Globally of 2-2.5% on China 0.7% outside
By age
- Below 40 years old 0.2%
- 50-60 years old 1.3%
- 60-70 years old 3.6%
- 70-80 years old 8%
- Above 80 years old 14.8%

Also compare with the information here, which takes it from this document used for GB decision mak ing.

Time from infection to disease, incubation time:

5-7 days Source: webminar

High risk groups are people that are more exposed to get the virus or have other comorbodities such as previous respiratory diseases or other major disease afecting the immune system (like cancer, HIV…) :

Comorbidities ?, we’ll use MorbiditySpainR to find it out.

As seen on the death rate, the age at the moment of the infection is important. Generally Italy and Spain has a population age older than China:

? We’ll use the data on INE thanks to INEbaseR

Modelling

One example of such analysis is this tweet:

Acabamos de evaluar la predicción del efecto de las políticas de restricción de movilidad. Equipo @jtmatamalas @SergioGomezJ @stinomat @urv y @gomezgardenes @claragranell @sorianopanos @wlcota @unizar. Restricción de movilidad total, excepto servicios esenciales, necesaria YA. pic.twitter.com/YaWqXHw7Qv
— Alex Arenas (@_AlexArenas) March 16, 2020

We could use models based on chapter 10 of the book Network Science of Albert-Lázló Barábasi, and from slide 28.

But follwing this advice I won’t:

To my colleagues, who are highly accomplished geniuses in a variety of fields: now is not the time to start an independent public health or infectious disease modeling practice if you have no expertise in these areas.

Please get an expert to check your work. Don't just post.
— Michael Hoffman (@michaelhoffman) March 18, 2020

However, here I leave a link to the UCI bed currently on use from DATADISTA.

Reproducibility

## ─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────
##  setting  value                       
##  version  R version 4.0.1 (2020-06-06)
##  os       Ubuntu 20.04.1 LTS          
##  system   x86_64, linux-gnu           
##  ui       X11                         
##  language (EN)                        
##  collate  en_US.UTF-8                 
##  ctype    en_US.UTF-8                 
##  tz       Europe/Madrid               
##  date     2021-01-08                  
## 
## ─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────
##  package     * version date       lib source                           
##  assertthat    0.2.1   2019-03-21 [1] CRAN (R 4.0.1)                   
##  blogdown      0.21.84 2021-01-07 [1] Github (rstudio/blogdown@c4fbb58)
##  bookdown      0.21    2020-10-13 [1] CRAN (R 4.0.1)                   
##  cli           2.2.0   2020-11-20 [1] CRAN (R 4.0.1)                   
##  crayon        1.3.4   2017-09-16 [1] CRAN (R 4.0.1)                   
##  digest        0.6.27  2020-10-24 [1] CRAN (R 4.0.1)                   
##  evaluate      0.14    2019-05-28 [1] CRAN (R 4.0.1)                   
##  fansi         0.4.1   2020-01-08 [1] CRAN (R 4.0.1)                   
##  glue          1.4.2   2020-08-27 [1] CRAN (R 4.0.1)                   
##  htmltools     0.5.0   2020-06-16 [1] CRAN (R 4.0.1)                   
##  knitr         1.30    2020-09-22 [1] CRAN (R 4.0.1)                   
##  magrittr      2.0.1   2020-11-17 [1] CRAN (R 4.0.1)                   
##  rlang         0.4.10  2020-12-30 [1] CRAN (R 4.0.1)                   
##  rmarkdown     2.6     2020-12-14 [1] CRAN (R 4.0.1)                   
##  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 4.0.1)                   
##  stringi       1.5.3   2020-09-09 [1] CRAN (R 4.0.1)                   
##  stringr       1.4.0   2019-02-10 [1] CRAN (R 4.0.1)                   
##  withr         2.3.0   2020-09-22 [1] CRAN (R 4.0.1)                   
##  xfun          0.20    2021-01-06 [1] CRAN (R 4.0.1)                   
##  yaml          2.2.1   2020-02-01 [1] CRAN (R 4.0.1)                   
## 
## [1] /home/lluis/bin/R/4.0.1/lib/R/library