Fun with Macros and Preprocessors

Hi all! Today I am going to describe some interesting things associated with Macros and Preprocessor. But, first of all let us understand the definition of a C preprocessor.

A C preprocessor is a macro preprocessor for C language. It provides the ability for the inclusion of header files, macro expansions, conditional compilation and line control. You must have definitely seen its most common usage which is inclusion of headers. Each time you write “#include <stdio.h>” in you C code, the preprocessor replaces it with the text of the file ‘stdio.h’ (The printf which we commonly use is defined in this file).

In Kernel codes, you will frequently find the occurrences of code pieces shown below:

#ifdef <condition-1>
header_inclusion
#elif defined <condition-2>
Other_header_inclusion
#endif

This implies that if the condition-1 is satisfied the “header_inclusion” is included else if condition-2 is satisfied then “Other_header_inclusion” is included.

There are various other instances of macro expansion seen in Kernel. I was recently working on drivers/scsi/ directory for fixing missing prototype warning where I noticed a particular set of warning in file drivers/scsi/be2iscsi/be_main.c.

drivers/scsi/be2iscsi/be_main.c:153:1: warning: no previous prototype for beiscsi_log_enable_disp’ [-Wmissing-prototypes]
drivers/scsi/be2iscsi/be_main.c:153:1: warning: no previous prototype for beiscsi_log_enable_change’ [-Wmissing-prototypes]
drivers/scsi/be2iscsi/be_main.c:153:1: warning: no previous prototype for beiscsi_log_enable_store’ [-Wmissing-prototypes]
drivers/scsi/be2iscsi/be_main.c:153:1: warning: no previous prototype for beiscsi_log_enable_init’ [-Wmissing-prototypes]

The interesting part of these warnings is that when you git grep any of the above symbols, you do not see any usage of these symbols in kernel. (I was intrigued why the file is complaining about a symbol which is not present in it :P )
However, when you dig the code a little bit more, you realize that these symbols are defined in the file but are hidden under macros.

For instance: beiscsi_disp_param in the file defines functions named beiscsi_*_disp. Then BEISCSI_RW_ATTR calls that. So, basically BEISCSI_RW_ATTR(log_enable, …) creates all those functions.
And hence the fix to solve all these warnings was to add “static” keyword to the various macros that are defining functions in that file. (Interesting isn’t it :D)

I hope it was fun reading about the preprocessor. Will keep you posted about more such intriguing things …Bye :)

Rashika

Missing Prototypes: Some Challenges

I will start this blog post with a wide smile on my face, mainly because of the appreciation to my work provided by the Kernel community. I would like to say “Thank you” for boosting my morale. :D However, there were many challenges which I had to face while eliminating the missing-prototypes warning. I would be elaborating on a few in this blog post.

a) Architecture Specific Prototypes
It is very common to find symbols in the kernel which have their prototype definition in architecture specific headers. So, in such cases, we should move it to an appropriate header file (an architecture-independent header file). The reason to do this is the fact that function definition stays in all kernel build (regardless of architecture) while the prototype definition becomes available only when re-build in that particular architecture.

The pattern is likely to be similar for __weak symbols. We don’t want to prototype them in arch-specific headers but only in some general header associated with the source file that defines the architecture-independent stub version.

b) Functions used in init/main.c
The file init/main.c calls “init” functions from all over the kernel, and doesn’t tend to properly use headers for them. Therefore, it is difficult to find an appropriate header file to move the prototype into.

There are two general strategies to solve this issue; either init/main.c has to include a hundred different headers that define only one function it needs (along with piles of other functions, in some cases), or there should be a dedicated header. The latter seems to be a better option. Therefore, creating an include/linux/initmain.h ,and putting all the prototypes for the unprototyped extern’d init functions from init/main.c in there is a right choice. Then init/main can just include that header file, so can every .c file defining one of those init functions.

I will keep you all posted about the other interesting things which I come across while eliminating the warnings. Till then, bye :)

- Rashika

Git Tips

When you send a patch-set to the Linux Kernel’s Upstream, the reviews might include some suggestions that need to be altered in few specific patches. This requires moving your git commit history to a specific patch and correcting it. This, if not done correctly might lead to an increased amount of work. Today, in this blog post, I am going to share some Git tips which I frequently use to tackle the same issue.

The foremost thing is to work in branches. You should always dedicate a branch for a specific type of work. This helps you to reduce the overhead of moving back a lot to fix a particular patch. A new branch can be created using the following command:
git checkout -b <branch-name>

You can list all the branches present in your working directory using:
git branch -a

Once you feel that a branch’s work is done and it is rendered useless, you can easily delete it using:
git branch -d <branch-name>

You can also list the commit log of your current branch with the help of following command
git log --pretty=oneline --abbrev-commit

Now, moving forward to fixing a specific patch which involves re-basing. Git offers an interactive Re-base mechanism to fix specific patches. This can be done using the following commands:

Consider a case, where you want to fix a particular commit (say bbc569ef). You can move back to this particular commit using:
git rebase --interactive bbc569ef^

In the default editor, modify ‘pick’ to ‘edit’ in the line whose commit you want to modify. Make your changes and then stage them with
git add <file-name>

Now you can use
git commit --amend

to modify the commit and
git rebase --continue

to return back to the previous head commit.

These few tips help me a lot while working with the patch traffic. Hope this helps you all as well.

Rashika

Making the Kernel free of “missing-prototypes” – 2

Hello all. In this blog post, I will discuss about the hurdles that I faced while eliminating “missing-prototypes” warnings from the drivers/ section of the Kernel.

One major issue I faced frequently was choosing the correct header file. As mentioned in the previous post, while removing the warning for a particular symbol, if there are more than one files which use it then we move the prototype declaration to an appropriate header file.

There can be various ways of choosing a header file:-

- Pick a header which all the files(using the symbol) include and move the prototype declaration to that header file. (But there might be large number of files which are common, how to choose among them?)

- Pick a file which has the structure definitions of the parameters used by the function to remove the overhead of managing nested includes. (It might sometimes lead you to choose a more general header when you should use a more specific one?)

- Pick a header in which the function logically fits, such as one that already includes similar functions (especially other functions from the same source file). (But, the functions in the source file may have the function prototypes in more than one file.)

Not so simple, eh? :P However, according to me the last method for choosing the header is the most appropriate one because it helps to choose a more specific header rather than a general one.

Another issue points out at proper handling of nested includes. Sometimes, when you move a prototype declaration to an *appropriate* header, you might need to resolve header dependencies because the function arguments may use a structure defined in other header.

There are two solutions to this problem: either you include both the headers in all the files which contains that symbol; or you can include the required header in the header file you chose for moving the prototype declaration into. The latter makes the header file self-contained (i.e. if something in a header needs another header, the first header should include the second directly, so that a source file including the header doesn’t have to include additional headers.). It also makes it easier for maintainers to include the header files in the future. (else you need to spend time to understand the header dependencies and include all necessary files).

This week, I also got stuck in one particularly interesting puzzle which points out to the necessity to remove the “missing-prototypes” warning. I was trying to eliminate these warnings for drivers/gpu/ and noticed that radeon_reg.h and radeon_drv.h conflict due to incompatible definitions therefore making it difficult to add both the headers in a single file. If it was included, then it throws the following error:

drivers/gpu/drm/radeon/radeon_drv.h:849:0: warning: “RADEON_RB3D_DSTCACHE_CTLSTAT” redefined [enabled by default]
drivers/gpu/drm/radeon/radeon_reg.h:1627:0: note: this is the location of the previous definition
drivers/gpu/drm/radeon/radeon_drv.h:1057:0: warning: “R300_CP_RESYNC_ADDR” redefined [enabled by default]
drivers/gpu/drm/radeon/radeon_reg.h:3342:0: note: this is the location of the previous definition
drivers/gpu/drm/radeon/radeon_drv.h:1058:0: warning: “R300_CP_RESYNC_DATA” redefined [enabled by default]
drivers/gpu/drm/radeon/radeon_reg.h:3343:0: note: this is the location of the previous definition

I was therefore not able to add prototypes to radeon.h (which includes radeon_reg.h) or radeon_drv.h if a file that needs to include one already includes the other.

For eg. If I add the prototype declaration of function r600_cs_legacy_get_tiling_conf to either header file radeon.h or radeon_drv.h, then I have to add the corresponding header file to r600_cp.c or r600_cs.c. Hence, either r600_cp.c or r600_cs.c would end up including both.

This can be resolved by either removing the redundant and conflicting definitions from either radeon_reg.h or radeon_drv.h or making the definitions match so that the compiler does not complain.While this problem stands unresolved for now (waiting for maintainers’ response), but it does help us to notice certain minor nits that remain unnoticed in the kernel.

Woof!! It was interesting to tackle these challenges :) But, there are many more to go.
Will keep you posted about more…Bye :)

Rashika

Making the Kernel free of “missing-prototypes”

As I mentioned in my last blog-post, I began working on my project “Tree-wide warning fixes and static analysis enhancements”. I started with the task of elimination of all instances of the warning:

warning: no previous prototype for 'name_of_symbol' [-Wmissing-prototypes]

The kernel build log has many such occurrences. This warning is a non-default GCC warning. So, the question arises:

question How do you generate these warnings?

The answer is simple. You just need to make a little change to your Makefile i.e. add -Wmissing-prototypes to KBUILD_CFLAGS in the top-level Makefile and do a build with defconfig or allyesconfig (If -Wmissing-prototypes is specified, it tells the compiler to detect global functions that are defined without a previous prototype declaration).
Voila!, you will notice a pile of new warnings about symbols with no prototype declarations. Alternatively, you can build the kernel with W=1.

One more thing which is important while building a kernel, is to save the log (This will help you to deal with the log conveniently). To do this, use the following command:

make 2>&1 | tee /path/to/build.log

question Why this warning?

These warnings are particularly useful to remove unused code from the kernel and thus making the kernel much smaller for everyone. These also help to detect any ambiguity related to prototype type mismatch which is a real bug and can even break the kernel build.

question How to eliminate it?

There are various ways of removing this warning. Umm, before that we need to learn one more thing, which is to find the occurrences of a particular symbol. This is easily done by the powerful tool GIT, the command being:

git grep -w <symbol-name>

Alternatively, you can find a header file containing that symbol using:

git grep -w <symbol-name> -- '*.h'

Then for each symbol, check if the symbol is used elsewhere in the kernel, and apply the appropriate fix:

- If it isn’t used anywhere else, mark it “static”.
– If it is defined and not used anywhere else, remove the function definition.
– If it is used elsewhere and has a prototype in a header file, include that header file in the source file.
– If it is used elsewhere and doesn’t have a prototype in a header file, typically because the callers elsewhere use an explicit prototype in a .c file, then you should either find an appropriate header file to put the prototype in, create one, or as a last resort add a prototype right above the definition to silence the warning.

After making these changes, you submit the edited portion of the file(s) in the form of a patch to appropriate recipients as indicated by get_maintainer.pl script. This script informs you about whom the patches have to be send to and why. To learn more about sending patches to the kernel’s mailing list, you can take the help of this easy-to-follow tutorial Linux Kernel: Submit a patch.

For now, I plan to eliminate *all* these instances by the next time I write a blog-post :D and move on to test these warnings on different architectures.
I will keep you posted about more such interesting things which I will keep learning during my internship. Till then, good-bye :)

Rashika

Open Source World: An Odyssey of discovery

Have you ever found yourself facing a wall of hesitation because you don’t know how to effectively start your intended blog post? Do you find yourself staring at the computer screen for several hours mainly because you are clueless on how to connect all your ideas into a cohesive start?

Well, I do feel the same, so, I thought why not begin with this :P

Let me introduce myself first. I am Rashika Kheria, a CS Undergraduate student. Umm, a better introduction would be Rashika Kheria, OPW Intern working on Linux Kernel. Doesn’t it sound great? :D Now, you must be wondering that how all of a sudden, I became an OPW intern, well, the credit goes to GNOME‘s OPW program, the supportive and the helpful Linux community and a little bit to me ( :P ).

I was always intrigued to work and contribute to FOSS projects, but I never got an opportunity to actually dive into it (courtesy: my laziness) and FINALLY, I got to hear about the OPW December-March program, so I thought,”let’s give it a whirl”. I surfed the web and tried to find everything about it. I went through all the participating organizations, the projects offered by them and finally decided to work for the Linux community.

When I started with the application period on October 1st, I had no idea about what I was going to do. But eventually, to my surprise, it turned out to be very enjoyable, interesting, addictive and full of learning experience. My routine became wake up, change the driver’s code, submit patches, get the reviews, modify the patches and sleep (The worst I can tell is that I started dreaming about patches :P ). I did not realize when the application period got over and I started waiting for the results.

The results were coming on November 25, 12:30 a.m. (Technically, November 26 in India). I was trying to avoid thinking about it to lessen my anxiety. But my attempt finally failed at 8:00 p.m. and I stopped studying for my University exam (scheduled on November 27 :P ). I started refreshing the page on which the result was going to be announced from 12:15 a.m. and as soon as it came out, I shouted with glee, “Yes!! My name is there. I am selected as an OPW intern”. This feeling was so awesome :D . This was followed by a series of mails instructing the selected interns about what to do next. And this attempt of writing a blog post is one of those :P

Right now, I am excited to work on my project “Tree-wide warning fixes and static analysis enhancements” and officially start with my internship. I will keep you all updated about the exciting new things that I am going to learn in this Odyssey :)

Rashika