Discussion:
[yocto] [meta-raspberrypi] Adding an additional device tree overlay in the SDCard
Herve Jourdain
2015-07-30 03:27:03 UTC
Permalink
Hello,



I'm trying to add a specific .dts file to the raspberrypi.

Therefore, in my own layer, I have made a linux-raspberrypi_3.%.bbappend,
that applies a patch to the arch/arm/boot/dts makefile to add my own layer,
and a new rule that copies my specific .dts file to the kernel source
directory.

And, initially, I had also defined KERNEL_DEVICETREE += "myown-overlay.dtb"
in there.



It definitely compiles well, installation process is smooth, I get the
Image-myown-overlay.dtb in tmp/deploy/raspberrypi2.



BUT when I flash my SDCard, it does NOT appear in the "overlays" directory
of the DOS partition!



If I add it manually, everything works fine, so I know my overlay has been
correctly generated and is working.



What I discovered is that, the sdcard_image_rpi class takes the list of .dtb
to be considered from split_overlays(), which is defined in
linux-raspberrypi-base class.

And split_overlays() basically gets the value of KERNEL_DEVICETREE (dts =
d.getVar("KERNEL_DEVICETREE", True)).



But when I set KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend, it seems
the additional value it defines is NOT taken into account when
linux-raspberry-class evaluates the value of KERNEL_DEVICETREE.

I've checked in the create_image.rpi-sdimg script created, it does not
appear in DTS nor DT_OVERLAYS.



But when I define KERNEL_DEVICETREE in my distro.conf, THEN it is taken into
account.



My goal would be to add that overlay by NOT modifying distro.conf (distros
are not necessarily RaspberryPi centric) nor in the raspberry machine files
(or local.conf).

This leads me to several questions:

#1 Is there a specific reason why my addition to KERNEL_DEVICETREE gets lost
when creating the image? I mean, is it normal, or are there some steps I
missed there?

#2 Is there a way that I could achieve adding my .dtb overlay in the sdcard
image only from my linux kernel bbappend?

#3 On a more general note, wouldn't it be simpler (better?) to just add all
the .dtb that are found in the deploy/images/ directory to the DOS partition
of the SDCard? After all, the .dtb need to be compiled from a .dts, so all
the .dtb that appear in deploy/images are "legit".



Herve
Petter Mabäcker
2015-07-31 13:29:20 UTC
Permalink
Hi Herve,

See inline answers.

BR,
Petter
Post by Herve Jourdain
Hello,
I’m trying to add a specific .dts file to the raspberrypi.
Therefore, in my own layer, I have made a
linux-raspberrypi_3.%.bbappend, that applies a patch to the
arch/arm/boot/dts makefile to add my own layer, and a new rule that
copies my specific .dts file to the kernel source directory.
And, initially, I had also defined KERNEL_DEVICETREE +=
“myown-overlay.dtb” in there.
It definitely compiles well, installation process is smooth, I get the
Image-myown-overlay.dtb in tmp/deploy/raspberrypi2.
BUT when I flash my SDCard, it does NOT appear in the “overlays”
directory of the DOS partition!
If I add it manually, everything works fine, so I know my overlay has
been correctly generated and is working.
What I discovered is that, the sdcard_image_rpi class takes the list
of .dtb to be considered from split_overlays(), which is defined in
linux-raspberrypi-base class.
And split_overlays() basically gets the value of KERNEL_DEVICETREE
(dts = d.getVar("KERNEL_DEVICETREE", True)).
But when I set KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend, it
seems the additional value it defines is NOT taken into account when
linux-raspberry-class evaluates the value of KERNEL_DEVICETREE.
I’ve checked in the create_image.rpi-sdimg script created, it does not
appear in DTS nor DT_OVERLAYS.
But when I define KERNEL_DEVICETREE in my distro.conf, THEN it is
taken into account…
My goal would be to add that overlay by NOT modifying distro.conf
(distros are not necessarily RaspberryPi centric) nor in the raspberry
machine files (or local.conf).
#1 Is there a specific reason why my addition to KERNEL_DEVICETREE
gets lost when creating the image? I mean, is it normal, or are there
some steps I missed there?
Yes, in the current solution KERNEL_DEVICETREE is needed in both the
kernel recipe and image creation context. When appending
KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend your new overlay
will only be visible within the linux-raspberrypi scope and NOT within
the image class.
Post by Herve Jourdain
#2 Is there a way that I could achieve adding my .dtb overlay in the
sdcard image only from my linux kernel bbappend?
If I remember the code correct within the kernel scope we are only
interesting if device tree should be used. At least in theory (assuming
you are using a >= 3.18 kernel since then KERNEL_DEVICETREE will bve set
by default) you should be able to skip the KERNEL_DEVICETREE += code in
linux-raspberrypi_3.%.bbappend (since it will be set any way and the
overlays themselves isn't really used in here..) then append the
KERNEL_DEVICETREE within the image scope. If you don't want to put this
into the layer.conf or similiar in your own layer I guess it should work
to "prepend" for example the IMAGE_CMD. But in the current solution I
guess local.conf, layer.conf (in your own layer) or rpi-base.inc (in
meta-raspberrypi) is the easiest options..
Post by Herve Jourdain
#3 On a more general note, wouldn’t it be simpler (better?) to just
add all the .dtb that are found in the deploy/images/ directory to the
DOS partition of the SDCard? After all, the .dtb need to be compiled
from a .dts, so all the .dtb that appear in deploy/images are “legit”.
That might perhaps be a better solution. Have you tried it out? Andrei
or Francois might have some comments about this?
Post by Herve Jourdain
Herve
Francois Muller
2015-07-31 15:16:15 UTC
Permalink
Hi,
My comment below.
Best regards,
François
Post by Petter Mabäcker
Hi Herve,
See inline answers.
BR,
Petter
Post by Herve Jourdain
Hello,
I’m trying to add a specific .dts file to the raspberrypi.
Therefore, in my own layer, I have made a linux
-raspberrypi_3.%.bbappend, that applies a patch to the
arch/arm/boot/dts makefile to add my own layer, and a new rule that
copies my specific .dts file to the kernel source directory.
And, initially, I had also defined KERNEL_DEVICETREE += “myown
-overlay.dtb” in there.
It definitely compiles well, installation process is smooth, I get
the Image-myown-overlay.dtb in tmp/deploy/raspberrypi2.
BUT when I flash my SDCard, it does NOT appear in the “overlays”
directory of the DOS partition!
If I add it manually, everything works fine, so I know my overlay
has been correctly generated and is working.
What I discovered is that, the sdcard_image_rpi class takes the
list of .dtb to be considered from split_overlays(), which is
defined in linux-raspberrypi-base class.
And split_overlays() basically gets the value of KERNEL_DEVICETREE
(dts = d.getVar("KERNEL_DEVICETREE", True)).
But when I set KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend,
it seems the additional value it defines is NOT taken into account
when linux-raspberry-class evaluates the value of
KERNEL_DEVICETREE.
I’ve checked in the create_image.rpi-sdimg script created, it does
not appear in DTS nor DT_OVERLAYS.
But when I define KERNEL_DEVICETREE in my distro.conf, THEN it is
taken into account

My goal would be to add that overlay by NOT modifying distro.conf
(distros are not necessarily RaspberryPi centric) nor in the
raspberry machine files (or local.conf).
#1 Is there a specific reason why my addition to KERNEL_DEVICETREE
gets lost when creating the image? I mean, is it normal, or are
there some steps I missed there?
Yes, in the current solution KERNEL_DEVICETREE is needed in both the
kernel recipe and image creation context. When appending
KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend your new overlay
will only be visible within the linux-raspberrypi scope and NOT
within the image class.
Post by Herve Jourdain
#2 Is there a way that I could achieve adding my .dtb overlay in
the sdcard image only from my linux kernel bbappend?
If I remember the code correct within the kernel scope we are only
interesting if device tree should be used. At least in theory
(assuming you are using a >= 3.18 kernel since then KERNEL_DEVICETREE
will bve set by default) you should be able to skip the
KERNEL_DEVICETREE += code in linux-raspberrypi_3.%.bbappend (since it
will be set any way and the overlays themselves isn't really used in
here..) then append the KERNEL_DEVICETREE within the image scope. If
you don't want to put this into the layer.conf or similiar in your
own layer I guess it should work to "prepend" for example the
IMAGE_CMD. But in the current solution I guess local.conf, layer.conf
(in your own layer) or rpi-base.inc (in meta-raspberrypi) is the
easiest options..
If modifying *.inc / *.conf files is not an option, then I would give a
try to Petter's suggestion to create a new image recipe (or bbappending
the current one). I see no reason this shouldn't do the trick, while
still being easy.
Post by Petter Mabäcker
Post by Herve Jourdain
Post by Herve Jourdain
#3 On a more general
note, wouldn’t it be simpler (better?) to just add all the
.dtb that are found in the deploy/images/ directory to the
DOS partition of the SDCard? After all, the .dtb need to be
compiled from a .dts, so all the .dtb that appear in
deploy/images are “legit”.
That might perhaps be a better solution. Have you tried it out?
Andrei or Francois might have some comments about this?
As far as I can remember, deploy/images/ links are only updated (meaning created if not previously present, updated if a new artefact is available) but never removed. This implies that if you simply copy 'deploy/images/*-overlay.dtb' to the sdcard, your image may end up with some devicetree overlays that are potentially no longer wanted or (even worse) not to be used with the specific kernel that will boot up the board...
That's the reason why I think choosing consciously the devicetree (overlay) set to pack in the sdcard may be important.
Post by Petter Mabäcker
Post by Herve Jourdain
Post by Herve Jourdain
Herve
Herve Jourdain
2015-08-01 01:34:20 UTC
Permalink
Hi François,



Well, if the .dtb might not get removed, like you describe, then changing the sdcard_image-rpi.bbclass the way I do might for sure create issues at some point.

It works for me – obviously – but definitely can’t be generalized.

Thanks for the insight :)



Best regards,



Herve



From: Francois Muller [mailto:***@gmail.com]
Sent: vendredi 31 juillet 2015 17:16
To: Petter MabÀcker <***@technux.se>; Herve Jourdain <***@neuf.fr>; Andrei Gherzan <***@gherzan.ro>
Cc: ***@yoctoproject.org
Subject: Re: [yocto] [meta-raspberrypi] Adding an additional device tree overlay in the SDCard



Hi,



My comment below.



Best regards,

François



Le vendredi 31 juillet 2015 à 15:29 +0200, Petter MabÀcker a écrit :

Hi Herve,

See inline answers.

BR,
Petter

On 07/30/2015 05:27 AM, Herve Jourdain wrote:

Hello,



I’m trying to add a specific .dts file to the raspberrypi.

Therefore, in my own layer, I have made a linux-raspberrypi_3.%.bbappend, that applies a patch to the arch/arm/boot/dts makefile to add my own layer, and a new rule that copies my specific .dts file to the kernel source directory.

And, initially, I had also defined KERNEL_DEVICETREE += “myown-overlay.dtb” in there.



It definitely compiles well, installation process is smooth, I get the Image-myown-overlay.dtb in tmp/deploy/raspberrypi2.



BUT when I flash my SDCard, it does NOT appear in the “overlays” directory of the DOS partition!



If I add it manually, everything works fine, so I know my overlay has been correctly generated and is working.



What I discovered is that, the sdcard_image_rpi class takes the list of .dtb to be considered from split_overlays(), which is defined in linux-raspberrypi-base class.

And split_overlays() basically gets the value of KERNEL_DEVICETREE (dts = d.getVar("KERNEL_DEVICETREE", True)).



But when I set KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend, it seems the additional value it defines is NOT taken into account when linux-raspberry-class evaluates the value of KERNEL_DEVICETREE.

I’ve checked in the create_image.rpi-sdimg script created, it does not appear in DTS nor DT_OVERLAYS.



But when I define KERNEL_DEVICETREE in my distro.conf, THEN it is taken into account




My goal would be to add that overlay by NOT modifying distro.conf (distros are not necessarily RaspberryPi centric) nor in the raspberry machine files (or local.conf).

This leads me to several questions:

#1 Is there a specific reason why my addition to KERNEL_DEVICETREE gets lost when creating the image? I mean, is it normal, or are there some steps I missed there?

Yes, in the current solution KERNEL_DEVICETREE is needed in both the kernel recipe and image creation context. When appending KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend your new overlay will only be visible within the linux-raspberrypi scope and NOT within the image class.




#2 Is there a way that I could achieve adding my .dtb overlay in the sdcard image only from my linux kernel bbappend?


If I remember the code correct within the kernel scope we are only interesting if device tree should be used. At least in theory (assuming you are using a >= 3.18 kernel since then KERNEL_DEVICETREE will bve set by default) you should be able to skip the KERNEL_DEVICETREE += code in linux-raspberrypi_3.%.bbappend (since it will be set any way and the overlays themselves isn't really used in here..) then append the KERNEL_DEVICETREE within the image scope. If you don't want to put this into the layer.conf or similiar in your own layer I guess it should work to "prepend" for example the IMAGE_CMD. But in the current solution I guess local.conf, layer.conf (in your own layer) or rpi-base.inc (in meta-raspberrypi) is the easiest options..



If modifying *.inc / *.conf files is not an option, then I would give a try to Petter's suggestion to create a new image recipe (or bbappending the current one). I see no reason this shouldn't do the trick, while still being easy.







#3 On a more general note, wouldn’t it be simpler (better?) to just add all the .dtb that are found in the deploy/images/ directory to the DOS partition of the SDCard? After all, the .dtb need to be compiled from a .dts, so all the .dtb that appear in deploy/images are “legit”.


That might perhaps be a better solution. Have you tried it out? Andrei or Francois might have some comments about this?



As far as I can remember, deploy/images/ links are only updated (meaning created if not previously present, updated if a new artefact is available) but never removed. This implies that if you simply copy 'deploy/images/*-overlay.dtb' to the sdcard, your image may end up with some devicetree overlays that are potentially no longer wanted or (even worse) not to be used with the specific kernel that will boot up the board...

That's the reason why I think choosing consciously the devicetree (overlay) set to pack in the sdcard may be important.





Herve
Herve Jourdain
2015-08-01 01:22:30 UTC
Permalink
Hi Petter,



Thanks for your answers.

At some point, I’ve set it in the disto.conf – it works there – but for
maintenance reasons I’d love to be able to keep it “localized” with the
kernel .bbappend – since it clearly depends on the kernel.



For now, I’ve just patched the sdcard_image-rpi.bbclass, to take the
existing ${KERNEL_IMAGE}*.dtb in the ${DEPLOY_DIR_DIRECTORY}, split them in
overlay and non-overlay, and then do the processing like it’s done now.

It does work in my case, and I can provide the patch if it’s deemed of
interest. But I’m not sure whether or not there can be some side effects –
looks like rather safe, but more opinions are definitely needed on that.



Regarding the kernel, yes I’m using the 3.18 that is supported by the
meta-raspberry layer. As I mentioned in another thread, with more recent
kernels, the RaspberryPi overlay layers were added in a separate directory,
so in addition to a patch to build the newer kernel, the
sdcard_image-rpi.bbclass will also have to be modified to get the files in
the proper directory, when we switch to those newer kernels.



Best regards,



Herve



From: Petter Mabäcker [mailto:***@technux.se]
Sent: vendredi 31 juillet 2015 15:29
To: Herve Jourdain <***@neuf.fr>; ***@concept-embarque.fr;
Andrei Gherzan <***@gherzan.ro>
Cc: ***@yoctoproject.org
Subject: Re: [yocto] [meta-raspberrypi] Adding an additional device tree
overlay in the SDCard



Hi Herve,

See inline answers.

BR,
Petter

On 07/30/2015 05:27 AM, Herve Jourdain wrote:

Hello,



I’m trying to add a specific .dts file to the raspberrypi.

Therefore, in my own layer, I have made a linux-raspberrypi_3.%.bbappend,
that applies a patch to the arch/arm/boot/dts makefile to add my own layer,
and a new rule that copies my specific .dts file to the kernel source
directory.

And, initially, I had also defined KERNEL_DEVICETREE += “myown-overlay.dtb”
in there.



It definitely compiles well, installation process is smooth, I get the
Image-myown-overlay.dtb in tmp/deploy/raspberrypi2.



BUT when I flash my SDCard, it does NOT appear in the “overlays” directory
of the DOS partition!



If I add it manually, everything works fine, so I know my overlay has been
correctly generated and is working.



What I discovered is that, the sdcard_image_rpi class takes the list of .dtb
to be considered from split_overlays(), which is defined in
linux-raspberrypi-base class.

And split_overlays() basically gets the value of KERNEL_DEVICETREE (dts =
d.getVar("KERNEL_DEVICETREE", True)).



But when I set KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend, it seems
the additional value it defines is NOT taken into account when
linux-raspberry-class evaluates the value of KERNEL_DEVICETREE.

I’ve checked in the create_image.rpi-sdimg script created, it does not
appear in DTS nor DT_OVERLAYS.



But when I define KERNEL_DEVICETREE in my distro.conf, THEN it is taken into
account…



My goal would be to add that overlay by NOT modifying distro.conf (distros
are not necessarily RaspberryPi centric) nor in the raspberry machine files
(or local.conf).

This leads me to several questions:

#1 Is there a specific reason why my addition to KERNEL_DEVICETREE gets lost
when creating the image? I mean, is it normal, or are there some steps I
missed there?

Yes, in the current solution KERNEL_DEVICETREE is needed in both the kernel
recipe and image creation context. When appending KERNEL_DEVICETREE in
linux-raspberrypi_3.%.bbappend your new overlay will only be visible within
the linux-raspberrypi scope and NOT within the image class.




#2 Is there a way that I could achieve adding my .dtb overlay in the sdcard
image only from my linux kernel bbappend?


If I remember the code correct within the kernel scope we are only
interesting if device tree should be used. At least in theory (assuming you
are using a >= 3.18 kernel since then KERNEL_DEVICETREE will bve set by
default) you should be able to skip the KERNEL_DEVICETREE += code in
linux-raspberrypi_3.%.bbappend (since it will be set any way and the
overlays themselves isn't really used in here..) then append the
KERNEL_DEVICETREE within the image scope. If you don't want to put this into
the layer.conf or similiar in your own layer I guess it should work to
"prepend" for example the IMAGE_CMD. But in the current solution I guess
local.conf, layer.conf (in your own layer) or rpi-base.inc (in
meta-raspberrypi) is the easiest options..




#3 On a more general note, wouldn’t it be simpler (better?) to just add all
the .dtb that are found in the deploy/images/ directory to the DOS partition
of the SDCard? After all, the .dtb need to be compiled from a .dts, so all
the .dtb that appear in deploy/images are “legit”.


That might perhaps be a better solution. Have you tried it out? Andrei or
Francois might have some comments about this?





Herve
Petter Mabäcker
2015-08-01 21:44:15 UTC
Permalink
Hi,

BR,
Petter
Post by Herve Jourdain
Hi Petter,
Thanks for your answers.
At some point, I’ve set it in the disto.conf – it works there – but
for maintenance reasons I’d love to be able to keep it “localized”
with the kernel .bbappend – since it clearly depends on the kernel.
For now, I’ve just patched the sdcard_image-rpi.bbclass, to take the
existing ${KERNEL_IMAGE}*.dtb in the ${DEPLOY_DIR_DIRECTORY}, split
them in overlay and non-overlay, and then do the processing like it’s
done now.
It does work in my case, and I can provide the patch if it’s deemed of
interest. But I’m not sure whether or not there can be some side
effects – looks like rather safe, but more opinions are definitely
needed on that.
Feel free to either send a RFC patch to the mailinglist or start by
sending it for early feedback to me, Andrei and Alex.
Post by Herve Jourdain
Regarding the kernel, yes I’m using the 3.18 that is supported by the
meta-raspberry layer. As I mentioned in another thread, with more
recent kernels, the RaspberryPi overlay layers were added in a
separate directory, so in addition to a patch to build the newer
kernel, the sdcard_image-rpi.bbclass will also have to be modified to
get the files in the proper directory, when we switch to those newer
kernels.
I remember that thread. We are currently working with integrating 4.x
into meta-raspberrypi you can follow Alex work at
http://redmine.gherzan.com/issues/62. One problem is however how to
handle support for both 3.x and 4.x with the current KERNEL_DEVICETREE
solution in combination with the changed overlay directories.
Post by Herve Jourdain
Best regards,
Herve
*Sent:* vendredi 31 juillet 2015 15:29
*Subject:* Re: [yocto] [meta-raspberrypi] Adding an additional device
tree overlay in the SDCard
Hi Herve,
See inline answers.
BR,
Petter
Hello,
I’m trying to add a specific .dts file to the raspberrypi.
Therefore, in my own layer, I have made a
linux-raspberrypi_3.%.bbappend, that applies a patch to the
arch/arm/boot/dts makefile to add my own layer, and a new rule
that copies my specific .dts file to the kernel source directory.
And, initially, I had also defined KERNEL_DEVICETREE +=
“myown-overlay.dtb” in there.
It definitely compiles well, installation process is smooth, I get
the Image-myown-overlay.dtb in tmp/deploy/raspberrypi2.
BUT when I flash my SDCard, it does NOT appear in the “overlays”
directory of the DOS partition!
If I add it manually, everything works fine, so I know my overlay
has been correctly generated and is working.
What I discovered is that, the sdcard_image_rpi class takes the
list of .dtb to be considered from split_overlays(), which is
defined in linux-raspberrypi-base class.
And split_overlays() basically gets the value of
KERNEL_DEVICETREE (dts = d.getVar("KERNEL_DEVICETREE", True)).
But when I set KERNEL_DEVICETREE in
linux-raspberrypi_3.%.bbappend, it seems the additional value it
defines is NOT taken into account when linux-raspberry-class
evaluates the value of KERNEL_DEVICETREE.
I’ve checked in the create_image.rpi-sdimg script created, it does
not appear in DTS nor DT_OVERLAYS.
But when I define KERNEL_DEVICETREE in my distro.conf, THEN it is
taken into account…
My goal would be to add that overlay by NOT modifying distro.conf
(distros are not necessarily RaspberryPi centric) nor in the
raspberry machine files (or local.conf).
#1 Is there a specific reason why my addition to KERNEL_DEVICETREE
gets lost when creating the image? I mean, is it normal, or are
there some steps I missed there?
Yes, in the current solution KERNEL_DEVICETREE is needed in both the
kernel recipe and image creation context. When appending
KERNEL_DEVICETREE in linux-raspberrypi_3.%.bbappend your new overlay
will only be visible within the linux-raspberrypi scope and NOT within
the image class.
#2 Is there a way that I could achieve adding my .dtb overlay in
the sdcard image only from my linux kernel bbappend?
If I remember the code correct within the kernel scope we are only
interesting if device tree should be used. At least in theory
(assuming you are using a >= 3.18 kernel since then KERNEL_DEVICETREE
will bve set by default) you should be able to skip the
KERNEL_DEVICETREE += code in linux-raspberrypi_3.%.bbappend (since it
will be set any way and the overlays themselves isn't really used in
here..) then append the KERNEL_DEVICETREE within the image scope. If
you don't want to put this into the layer.conf or similiar in your own
layer I guess it should work to "prepend" for example the
IMAGE_CMD.But in the current solution I guess local.conf, layer.conf
(in your own layer) or rpi-base.inc (in meta-raspberrypi) is the
easiest options..
#3 On a more general note, wouldn’t it be simpler (better?) to
just add all the .dtb that are found in the deploy/images/
directory to the DOS partition of the SDCard? After all, the .dtb
need to be compiled from a .dts, so all the .dtb that appear in
deploy/images are “legit”.
That might perhaps be a better solution. Have you tried it out? Andrei
or Francois might have some comments about this?
Herve
Loading...